DOCtRBVt IBSOBB 



8D 181 260 

iOTHOB 
TITIB . 

INSTITOTIOM 

BEPORT NO 
pod DATE 
HO^E • ^ 

EDBS PRICE 
DESCRIPTORS' 



CM P2 3 772 



\ 



Zajkowski* H. Hicha«l: And Ot.h«TS. 

injisMssMBt of Individualized Instruction in Navy 

Technical Traininq. 

Naval Training JinalysiB and *B valuation (iroup, 

orlandcfTv BlA- r 
TABG-B-7S 
HOV 79 • . 

B7p. • • 



llPOI/PCOU^Plua PoAaqa, 



IDENTIFIERS 



ABSTRACT 



Coaputer Assisted^ In&truct ion: *CoBputer Managed 
Inatruction: ♦Individ ua 11 xed Instruction: ♦Military 
Training; Prograacd Instruction: ♦Prqgraa 
Effectiveness^ ♦Progrl^li Evaluation: ♦Prograa 
■proveaent ~' 
Navy: Onlted States 



A study was con^ducted to estajbllsb the current status 
of Individual i2e(l Instruction (IT) in the Navf^f and other services^ 
identify the factors influencing its effectiveness, identify present 
or potential probles areas, and reconiend strategies /policies to 
isprove II .in||avy technical trainlno,^ Particular attention was given 
to an assessB^fkof instructor sanageaent of 11 by cospvter: F^st, 
all relevant Naif instructions, directives, a-nd guidance' sere 
reviewed and an asses ssent aade of ttheir ispact on tlv,e isplesentation 
and aanagesent of II in the Navy. Next, key sunsary articles dealing 
*with the effectiveness/efficiency of II were reviewed in ,an atteap^^w^ 
to establish a consensus concerning the utility of #this instructional 
strategy. Tlird, visits were sade to key sites in the IJavy and other 
silitalry services where inforsation ^rsuanrt to the establishsent of 
a cosparatlve data<base on II was obiiined. Finally, findings and 
recoasendations were de^loped on a 1\ inforsation^ obtained. The sajor 
conclusion of the study was that the Naval Education \and Training 
CoBsand sJiould continue to use II In Navy technical tr^ning. 
Specific recoBsendations for the. laproveMnt of II were provided in 
the .areas o| aanagesent Inforsation syster&« training of, instructors ' 
and aanageri, tVrninology, student-instructor incentives, cost 
benefit analyses, ^nd further *study requlresents. (A\^hor/BM) 



i|niiAS««** **«*««««**♦♦*♦♦>•<♦ ♦♦♦♦♦♦.♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦♦<'** ''.^ 

♦ Reproductions supplied by EDBS are €he best that can be aade, ♦ 

♦ frosvthe original ^qcuaent. ♦ 
««i»««««««i»«*ee*«««««4i***4i*4»«* **************** ♦)!; 



i 

rsi 



CO 



. JAEG Report No. 

AN ASSESSMENT OF INDIVipUALIZED INSTRUCTION IN 
NAVY TECHNIC!^ TRAILING 

M. Michael Zajkowski 
Edwar4 A. Heidi; 
James M. Corey 
Dorothy V. Mew 
Gene S. Michel 1 



Training Analysis and Evaluation Group 



November 1979 



us OiP»«TM«NT OF MeAUTM 
f OUC*TIO<t • WILFARi 
NATIONAL INSTITUTB-Of 
eOUCATION - 

IMIS (MUllMfNI HAS -BttN HM-HO. 

0(K f o f xAt 11 Y AS wm ivrn i hom 

IMF rlHSONOH OHl.ANI7ATIONOHH.IN. 
AnNGIT POINTS U» V I F W OH O.PI N IONS 
ST ATI I) on NOT NFl I SSAHIl Y Ht PHt 
StMIOIIKlAl NAIlONAl INSTIIllTf 01 
KHKATION POSITION OH »'Ol KY 



3 

ERIC 



GOVERNMENT RIGHTS IN DATA STATEMENT ' 

Reproduction of- this publication in whole 
or in part Is permitted for any purpose 
of the United States Government. ' 





ALFRED F. SMODE. P-h.D.. Director, 
Training, Analysis and Evaluation, Group 




WORTH SCANLAND. Ph.D. 
Assistant Chief of Staff for 
Research and Instructional Systems 

Development * ^ 

Chief of JIaval Education and Training 



2 



ERIC 



UNCLASSIFIED 



StCUHlTV CL AtSirlCATION OF THIS PAOE fWh^\ Omim Bh9mrmd) 



REPORT DOCUMENTATION PAGE 



TAEG Report No, 78 



2 OOVT ACCCSIION NQ 



4 TITLe (0nd Subtltim) 

AN ASSESSMENT OF INDIVIDUALIZED INSTRUCTION IN 
NAVY TECHNICAL TRAIMNG 



y AuTHORr«> • 



M. Michael Zajkowski 
Edward A. Heidt 
James M. Corey 



Dorothy V. Mew 
Gene S. Michel i 



9 PCRrOf^MINO OROANIZATION NAME AND ADDRCSS 

Training Analysis and Evaluation Group^ 
Orlando, *FL 32813 



I I CONTROLUNG ornct NAMt AND APORCSS ' 



U MONITORING AOCNCY NAME A ADDRESV" ContrMIIng Oiljcm) 



READ INSTRUCTIONS 
BKFQRE COMPLETING FORM 



% RECIPIENT'S CATALOG NUMtCR 



S 1 YPt or REPORT h PERIOD COVERED 



• PERrORMlNO ORG REPORT MUMStf^ 

I Contract OR fiRANT NUMitRf*) 7 



.10. PROGRAM ELEMENT, PROJECT. T AiK 
AREA A WORK UNIT NUMBERS 



12 REPORT DATE 

November 1979 



IS NUMBER OE PAGES 

83 



l» SeCyRITY CLASS, fo/ Ihia rmporl) 

Ui^classified 

« • ' 

1S«. blECL ASSIFICATION/^ DOWNGRADING 
SCHEDULE . 



DISTRIBUTION STATEMENT^o/ thim Rmport) 



Approved forpublic release; distribution is unlimit^ 



7. DISTRIBUTION STATEMENT (of thm •brntrmd •ofararf In Jlloc* 30, U dUtmrmrM 



1 



16 5UPf»LEMENTARY NOTES 



\ 



19 KEY WORDS (Conllnuu on rmvmrmm •(<<• II n«c««««r)r and Idmnllty ftV bloc* numbar) 



Indi vi dualized • Instruct ion 
Computer Managed Instruction 
Computer Aided Instruction 



Programmed lystruction 
Self-Paced Instruction 



•Ida If'nVMaawy and 



20. AaSTNACT (ConUnum on ravaraa alrfa II nKocaaarr and Idmnttty by block liooibar/ 

A study was conducted to (1) establish the current^tatus of Individualized 
Instruction (II) in the Navy and other services, (?) identify th^ facters 
influencing its effectiveness , (3) identify present .or potential ■problem areas, 
and (4) recommend strategies/policies to improve II in Navy "technical training. 
The major conclusion of the stijdy was that the Naval Education and Training 
Command shoyld continue to useXlI in Navy technical training. Specific 
recommendatiohs for' the improvement of II are provided in the areas of 



DD , "ISr,, 1473 



EDITION OP 1 NOV IS OBtOLfeTC 

d/N 0K)3*014^6601 | 



'ilNCLASSIFIED - ^ . 

BECUr/Vy CUAftl^lCATION OP TNIS PAOt (Whm% Dmf Kntmfmd) 



UNCLASSIFIED 



20. management Information systems, training of Instructors and managers. 
. ^terminology. student-Instructor Incentives, cost benefit analyses, and 
further study requirements. 



) 



/ 



UNCLASSIFIED 



ERIC 



■ 4 : : • 



% 4 



TACG Ueport No. 78. 



I 



SUMMARY * 

'Despite th6 apparent increases in traininfl efficfency associated with 
individualized instruction (II) theret aVe still legitimate concerns in the 
Navy with the soundness of the concept in th^ military context, the manner In 
which instruction is provided, and the quality of ^aduates from II programs. 
Te put these concerns into perspective, .the Training Analysis and Eva uation 
tfroup was tasked t? conduct an assessment of II in Navy technical training. 
The study (1) £stab>ished the current status of II in the NJvy and other 
services. (2) ideiTtifled the factors influencing its effectiveness. (3) 
identified presenf and potential problem^areas. and (4) recommended actions 
to better articulate to decision makers the rational^ of Instructional Systems 
Development (ISD) as well as to optimize'the implementation of II. in the 
Navy. Particular attention was given to an assessment of the management or ^ 
II by instructors and by computer. Key findings and recommendations from the 
study are outlined below. •« 

MILITARY APPLICAT^fel OF fNDIVIDUALIIED" INSTRUCTION 

•A substantial commitment tias been made to' the use of II in the military 
services An indication of the extent of that gommitment is contained in. 
section II of this report, because of the short -time available for this study 
and the unavailability of certain classes of data, it was not possible to 
establish the full rahge of II use by the military services. However, it is 
clear that the use of II, in particular computer managed instruction (CMI). 
computer aided instruction (CAI), and progp-ammed instrdcti'^n (PI), is exten- 
sive and is increasing in technical training. 

The Navy commitgient to II is most visible in CMI. FY 78 data show an 
approximate student f\OB and throughput for CMI couKes of 7,000 and 65.000, 
r4sReotivel>. There are an additional 3,000 student AOB and 59.000 student 
throughput for H courses which are not computer managed. CAI usage in 
technical training indicates 350 to 400 student stations ingse'or planned. 
However, an additional difficulty exists in" establishing the full extent of 
CAI in technical training because of the variety of instructional applications 
of computers designated. a» CAI-.. Programmed Instruction is the primary instruc- 
tional fonnat in all forms of II. 

MAJOR INFLUEIKES. ON THE EFFECTIVENESS OF . INDIVIDUALIZED INSTRUCTION ' 

'-.4. - ' • ' ' • 

Only a limited amount of evidence is available to -describe the existence 
and operation of factors influencing the effectiveness and efficiency of II. 
Available, data generally indicate that II is as effective ^s conventional 
instruction (CI) in terms of end of course achievement scores and that the 
efficiency of il is superior to that of CI in terms of student time to complete 
'ins true ttlbn'. However, no tjseful data were ,found which addressed the relative 
effectiveness oj.l'l 'for different kinds of training tasks or for varying 
ability levels 'of trainees. . , ' s 

' A ndmber Oif factors which exBrt an influence on the effectiveness/efficiency 
il in Navy technical training. are identified and discussed in this study. 
These inQjude:. ' " ' : 
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The organizational structure supporting 
i n^?fs^w*rif'' ""''^ frequently than any other 'factor as having a 
significant Influence on the effectiveness of II. Difficulties 
assoc1at;ed with this factor jncludg the complexUy of the Curretit 

Slan'^^^a^'h^H'S^H"'"' ^" Integration an/coordlna on o^ 

^iTfiJ'r. **"^?^^i"9» Instructional system developrrient processes- 
and the perceived absence of accounta^ll It/ for specific tasks ' 

Attitudes. Feelings toward II are m1;ied. Mdst^tudents aooear to 
VI ev. II either positively or with Indifference. sX 1ns??Sc?or 

TLTiV Pf!?""?^ ^^"^ ^° " «s Ineffective and/o? Inefficient • 
a small coterie Is vehemently opposed to II. Most Important am^nn 

^satl?fL'?r '° "^^atlve attitudes toward?? ?" a S?s? 

satisfaction that results from the dIffeVence between expected and 

'ex^\" a?'c n^^t'raJn'l^s^T^^ ' 'f^^^ apprec'YHe -^^^'^m 'c ^'f ^ 
triWn!. 5^ changes In course content are major con- ' 

JacJoJc ^'r' dissatisfaction.. Add?[loSal 

factors affecting attitudes Include confusion in termfnoloav' 
changing roles of Instructors and students, a acJ^ unde?s;;u,H1n- 
stporb"'"'''°"^ ^^-tegi^. and a perc^iJeJ'a^rorllJet^t' ' 

Resources . -The primary impact of reduced resources has been on 
suppoFTTervices at the schoolhouse ahd there is a'widlly helS oer- 
cepti^ by lower echelon activities that thefe is a ack of resource 
and management support for Il^programs. Further, there inav be a 
significant waste of resources\resulting fromT ack of TnLara?inn ' 
?I act?vU1e]'' the.,POM/budg'et'cyc?es%^'soc'lI;:dT?t?; 

Base?. A comprehensive assessment of the ef fectiveness/pfflrionru 

by CNET. external appraisal is being urrdertaken 

'o l e^'"^n°^{^':^s^^ Tr^: The instructor's ana}or onager's ' 

Sf^sl^Utrtl^lale^en"^^^^^^ 

t«Hn; "'""ssio" of course-administration are 

testing policies, predicted completion time, cflurse loading. 
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hardware support, and Management Informa^qn Sys.tein^(MIS) re(!iuire- 

• ments. Holding time, remediation. Incentives, and*housing and • 
^messing are Included In the discussion of student mdnagement. 

Finally, the discussion of Instructor management includes con- ^, 
sideration of plowback'pollcies and collateral duties. 

CONCLUSIONS AND. RECOMMENDATIONS 

' The evidence'presented in this report strongly indicates support for the 

co/itinued i>se of ll as an instructional strategy in Navy technical training. • 

However, to enhance H • effectiveness and efficiency in terms of Navy goals, 
the following actions are- recommended: 

. . " establish a single office/activity with re^onsibi 1 i ty for the 
integration and coordination of all aspects of TI. 

■ develop an information package which would communicate the rationale, 
' ' philosopy, and implementation procedures apd policies associated 
with II and present to all NAVEDT«ACOM and major fleet activities. 

initiate, and support an effort to determine the relative effectiveness/ 
- * efficiency of II for different kinds of training tasks and ability 
leve.ls of trainees. ' 

develop appropriate data bases and record -Iceeping .pracedures to:.. 

establi.sh the types and extent of II in use throughout the 
Navy. 

. compare the' cost efficiency and training effectiveness of 

" inst»*uct1or1al strategies', management systems, and^ADP alternatives 

facilitate the adminU't ration of II, in -the Navy. ^ 

, develop /and. implement criteria for selecting among alternative 
iostr^uctiondl strategies ,. instructional management systems, and/or 

• . instructional media. - >' ' 

. ensi/re the use of standard 11 terminology throughout the NAVEDTR'^COM. 

ensure that the training fiipeline forJI instructors includes ^ 
materials appropriate " to their role as Learning Center SupQ.r:vlsor/ 
Instructor. Implement this material, on an intenm basis pending 
the delivery of instructor training curricula under development. 

develop and implement an II management course for' all. training 
administrator and school /course management personnel.. • , 

examine the desirability of providing prepara«tory materials on the 
use of computers in instruction for stud.epts and/or instructors. 

establish a program to identify incentives. and/or procedures Which- 
act to. improve student and instructor performance in an II environment 
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develop and Implement a MIS for tfie management of instructor 
personnel ind1v1<lual training locations! 
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*^ Over the past quarter century there has been a trend in instructional 

■ strategies toward the use of individualized instruction (II). Experience to 
* date has shown that, in general, II, is as effective as conventional instruc- 
tion (CI), its major advantage being that ave»*age traitiing tiflie is reduced 
compared with the various conventional ap^proaches. Although II is frequently 
identified as a singular concept or approach, often substantial variations in 
instructional strategy, instructional management, and instructional delivery^ 
are subsumed under this general category. 

A number of representative summary reports (Orlansky and String, 1979'; 
Northrop Corp.>, 1971 ; |jDckheed-C^l iforhia Co., 1971; Middleton, Papetti , and 
Michel i, 1974) have documented the advantages, complexities, and problems of 
II and have described in detail Icey issues „ associated with its implementation. 
The most prominent of these issues are student achievement, student attrition, 
training effectiveness, student and Instructor attitudes, cost benefits, 
and instructor functi6ns. Thus, despite the apparent increa6e in efficiency- 
, associated with II, there are still legitimate concerns *1n the Navy with i|;s 
implementation and conduct as well as with the quality, Of graduates from the 
programs. ' • y >^ 

The Chief of Naval Education and Training (CNET)^ tasked the Training 
Analysis and Evaluation Group (TAEG) to examine II in Navy training and to 
place the irfajor jssues into perspective. The guidance provided called for a 
quick response effort. In order to meet this requirement, the study was 
restricted to enlisted technical training and to an identification of broad 
issues, problems, and analyses. .The work was begun in June 1979 and com- 
pleted in October 1979. 

" STUDY* OBJECTIVES ; - . 

The object ivq^ of this study were to: (1) determine the current status 
of II in th^ Navy and the other military services, (2) identify the ^^tors 
influencing its effectiveness, (3) identify present or potential probtlmr^ 
areas, and (4) recommend strategies/policies to better articulate to decision* 
makers the rationale^of Instructional Systems Development (ISD) as well as to 
optimize the implementation of II in the Navy. Particular attention was 
given to an assessment of the management of II by instructors and by computer. 

BACKGROUND 

The history of II in the Navy is inextricably interwoven with the. imple- 
mentation of the systems approach to the design and management of training 
and with research and development in programmed and comput&jr a^ded instruc- , 
tion. A brief perspective on these interlocking -developments is provided 
. .here. • ^ , . ' 

^ CNET Itr Code N-53 of 22* Aug 1979. . 

V ^ • . 9 ■ 
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- In the late 1960s tfie systems approach to the dtslgn and martagement of . 
instrljctt^nal systems was receiving Increased attenf!on 1n the Department of 
Defense (DOD) and by the Chief of Naval Operations (CNO). This activity 
served to highlight Navy programs (Rundqulst. 1967) which had already begun* 
utilizing the systems approach to InstrucVlonal program design and which had • 
not only pointed out the need for but already had begun to Implement II: 
JjQ documents Issued during this, period (NAVPERS 93510t1 and BUPERSINST 
t5^0«l3) addressed the systems a ppVoach to training. The first simply noted ' 
the trend toward the systems approach, while the second outlined procedures 
DH^rSc^r approach to Instructional development and Indicated that all 
SUPERS Courses were to be designed In accordance with these procedures. 

. In 1971, the Assistant Chief for Education and Training (P^RS C-229) 
issued, a memorandum to training managers providing guidance and asking for a 
review. of programs and the submission of plans-for converting appropriate 

i"?!''!'^"^^ learning systems^n 1973. the Chief of Naval Training * 

- (CNT; stated that one of his major ob/%ctTves was "to restructure all tralnlnq 
programs in accordance with the latesft and best tenants of Instructional 

. technology, placing highest priority /on those programs determined to yield . 
maximum benefits in pipeline roduction" (Cagle. 1973). This statement was 

1h ^J?!^""^? issuance of CNETINST 1550.5, Maribh 1974. which established 
the CNET policy and doctrine for the central tzed/control of Instructional 
program development. In September 1974. the CNl/T Alp Man;iaNwas Issued. 
This manual contained the approved procedure fo^, pl^nlnrf. designing, develop- 
ing and managing Navy technical training; 1 .ej Instrucflonal Systl^s Develop 
ment (ISO). Instructional Systems Developmentiias b^en described as an 

planning and developing InstructTlonal programs which 
insure that personnel are taught the knowledges, skills, and attitudes essen, 
Jin M job performance (Hodak. MldbTeton. and Rankin. 1979). The C NTT 
AlO Manual also stated that the preferred Instructional strategy for all Navy 
training courses was II. NAVEDTRA 106A issued In 1975 (phase III p 24) 

^nstr^ur^yfon'^n'T^n^'-P'^'"^ (Individualization) -was the^ deferred mode of ^ ^ 
instruction in ISD courses, 

•» 

In April 1976^. CNEt announced the decision to establish the Instructional 
Program 5eve opment Centers^ (IPDC) at San. Diego and Great Lakes fi? c^ntJir 
ized instructional program development. In July 1978. CNET issued- NAVEDTRA 
110. ^an extension of NAVEDTRA 106A. which prescribed policy, procedures . 
and guidelines for -the analysis, design, and development of all Instructional 
prograins within the Naval Education and Training Command (NAVEDTRACOM) extent 
submarine training programs. Thus. tKe systems approach to instructional * 
development and II became solidly imbedded in TJav/ training. / 

Overlapping the ^valuation of ISD was th^ concurrent research. ^develop- 
ment, and implementation of-. computer- based trafining in the Navy. The evalua- 
tion of the Navy computer managed instruction (CMI) system is described by 
Hansen, Ross. Bowman, and Thurmond (1975) and is reviewed briefly here Its 
major historical antecedents -were the programmed" instruction movement of the 
1950s and early 1960s and the computer-based Instruction work of the 1960s— 
particularly that sponsored^by the Off ice of Naval Research (ONR). These 
events together with significant Interactions among the ONR. Navy Training 
Research Laboratory (NTRL). and Chief of NavaJ Air Technical Training (CNATT) 
were. key to the implementation of Navy CMI. 1966. the Assistant Secretary. 
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of Defense,. Manpower and Reserve Affairs (ASD M&RA) a'T located funds to 
initiate the CMl project; the project was begun by CNATT in July 1967* 
Subsequently, Navy advanced development objectives^ providea the major funding 
t»>rough direction of Chief of Naval Personnel (CHNAVPERS)'. The period from , 
1968 to 1970 was characterized by joint institutional developments involving 
(1) computer software to support the CMl system, (2) 'research on media selec- 
tion and preparation and coding of CMI instructional materials, and (3)., 
feasibility studies of computer a-ided instruction (CAI) in the CMI system. 
Approval of the CMI. system ^s an operational element in Navy training was 
sought in 1970. A cost justification study which supported this request and 
formulated the bisic rationale for the decision to go operational was spon- 
sored by GNET. This was approved by CNO on 5 February ]971, and, after some 
delays in obtaining resource support, automated data processing (ADP) equipment 
acquisition was begun. ^,Th^ fir^t course, Aviation Fundamentals, was officially 
implanented into the system in 1972, and in 1974, CNET and CNTT adopted CMI as 
a fo/mal component of the Navy training system (CNETINST 5260.1, CNTTINST 
5400. 7A). In 1975, a contract was let for ADP hardware and services. .Finally, 
CNET Decision Memorandum No. 2 (27 April 1976) integrated CMI with the plans 
for the redesign of courses by the ^PDCs. 

The brief historical review contained in the preceding paragraphs 'outlines 
the evolution of II in the Navy. It" provides a perspective for*a more complete 
understanding of the complex issues associated with II identified in this report 

APPROACH ' • 

, ■• -f ' ' ' 

There were four major comfionents to the approach used in this study, 
first, all relevant Navy instructions, directives, and guidance were reviewed 
ahd an assessment made of their impact on the implementation and management 
of II in the -Navy. Next, key summary articles dealing ifith the effectiveness/ 
efficiency of II. were reviewed in an attempt to establish a consensus concern- 
'Ing'the utility of this instructional strategy. Third, visits were made to 
key sites in the Navy and other military services where information pursuant 
to the establishment of a comparative data base on II was obtained. Finally, 
findings and recommendations weV^e developed on all information obtained. < 
This latter information was obtained primarily in interviews conducted on 
site. A list of commands and activities contacted is provided in appendix A. 

DEFINITIONS ' f ' * 

♦ 

Because accomplishment of study objectives required' precise t^rnrinology, 
the following definitions were established and, are used throughout the report. 
They a re, based on and are consis4:ent with current CNET (CNETINST 1500.12) 
definitions and reflect the distinctions between instructional strategies, 
instructional management systems, and instructional del i very" systems (media). 

Individualized Instruction (TI). An instructional strategy in which all 
learning activities are designed to accommodate individual differences in 
background, skill level, aptitudes, and cognitive styles. Individualized 
Instruction is characterized by the following attributes: 

releasing of time constraints. 

choice of Instructional media 

11 I4 
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Instruction adjusted to skill levels and learner charac- 
teristics. K^often employs programmed Instruction. 

• Conventional ^nstruct1or(4 (CI ) . An Instructional strategy In which- ' - 
jlearning activities are dlrecteTtoward a normative model of the target 
Is'^hamJeHzed'^b^'^^'^^ ^""^ "'"^^^^ delivered Ir^ a graup environment. It 

^ . ^ predetermined group pairing 

preselected nonvar.tant media . • 

predetermined nonvariant instruction, 
of thrqrl.D^^''^^^''''^'"' established, are employed with all members 

P^Q^j^l^ed instruction An instructionaT format which presents 

individuaTized^ materials in a sequence of small unit;s each of which requires 
.an immediate response from the trainee and which also provides the trainee 
with liitnediate knowledge of results 



Programmed Instruction Text. An instructional delivery system wh id 
employs programmed Instruction. 

Computer ^I'ded In strucp'on {CAIj. An instructional delivery system in 
which a computer system Is used, to provide instruction and where' there is an 
2h2nTrii? stimulus and reaction between the computer and trainee. 

svsJem s^rvP^^hnlh''^ 'h^'^''^ within, the host computer system, the computer 
system serves both a media and management function. * ; 

in ^. f^PButer Ma^^ An instructional management system 

?^ • ?/°7P"^^': IS employed to prescribe a series of ins-tructlonal materials 
for individual trainees. Usually .associated with If. it may include the 

vaHou '^^H[rfn'';h' counseling, and the'^seleh on of 

various media for the delivery jaf instruction. 

,,,. ,^"^^^"P? °r .Managed Instruction ilMlj. An instructional management 
system n whTcK the instructor prescfTB^ a series of instructional materials 
for individual trainees. It is usually associated with the delivery of I 
and may include the capability for record keeping, testing, counselinq and 
the selection of various media for the delivery of instruction ^' 

Instructional Systems Development (ISDj. A systematicSprocess (frame- 

conS the^velopllrand 
conduct of training. This process usually includes five phasesVanal vze 
design, develop, implement, an^ control. ^ <J^es reanalyze. 

ORGANIZATION OF THE ^PORT 

other ^secHfen^^°" introductory section, the report contains th'^ree 

other sectr»fens Section II summarizes the status of II in the militarv 
services, provides a brief overview of research bearing on II. and presents .a 

o 12 / 
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review o/^coriomic analyses of 11 In the Navy. Seotlon III contains the _ 
findings of the study regarding those factors influencing the bffectiverlesS/ 
efficiency of 11 in the Navy. Section IV contains conclusions concerning ■ 
trends in training, technology, and iijanpower which .may Influence Navy trailing 
The section alsfi contaTlns reconwendations for the improvement of II in the / 

N«^y- . . / . • . ^ . ' ; . Y / 

Several appendices are included in thisVeport. Appendi/ A mts the 

commands and activities visited; appendix B contains a model and an alqorithij 

for the 'economic jinjilysis of II in the Navy; appendix C contains a listing of 
Navy technical training courses which ^Toy II. ^ 
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SECTION II . 

THE- CURRENT ST/\TUS OF' INDIVIDUALIZED INSTRUCTION IN THE MILITARY SERVICES 

This section contains an overview of the current status of U in the • 
military services and summarizes relevant literature regarding its effective-, 
ness and/pr efficiency. No attempt is made to trace the developmental htstory 
and implementation of II in environments other than the military. The reader 
interested in the broader issues and applications of II is referred, for- ( - 
example, to "Skinner (19'68), Blaisdell (1973), O'Neal (1970), Robinson and 
Lautenschlager (1971), Abramson (1970), and MitzeT (1971). 

« 

THE EFFECTIVENESS OF INDIVIDUALIZED INSTRUCTION 

A significant portion of military trainingcesources has been devoted 
to II. Initially, this commitment was in thejrorm of/ progranmed texts. 
Gradually, however, more afid more aspects of ivistruct/ional delivery have 
been automated. At the present time, II seems* to be primarily identified 
with computer managed and/or computer aided instruction. This is unfortunate 
since this .focus on tlife automated aspects of II has affected the ability to 
identify and track other forms of II. In their comprehensive review of 
computer based instruction in military training, Orlansky and String (1979) 
identified four major categories 6fiflata re-flecting ei ther, training efficiency 
or. effectiveness of computer based instruction. These included achievement, 
time savings, attrition, and attitudes of students and instructors. 

^ In terms of course achievemeat, CAI was found to be superior to conven- 
tional instruction in 15 studies, inferior in one study, and 24 studies showed 
na d.i f ference. When compared with PI, CAI was found to be superior in one 
study out of five; ther^ was no difference in four studies. However, 
course achievement as a measure of the relative effectiveness of al t*ernati ve ' 
instructional strategies should be used with caution. It is inevitable 
that: few differences in achievement have been found since students remain 
under instruction irf CAI «nd CMI until they achieve standards equivalent to 
those set for CI. ^ . 

Time savings associated with CAI and CMI are dramatic when compared to 
CI. It was reported that CAI saves approximately 29 percent (median) with 
a ra^e reported of 10 to 89 percent. Thirty-six of 40 casfes reported a 
time savings, three reported 'increases in -course completion time and one 
reported no difference. Computer managed instruction (seven cases) is 
reported to save approximately 44 percent (median) in course time with a 
range of 31 to 89 percent. When CAI and CMI were combined in a single 
program, savings bf 32 percent (median) in course time were obtained. 

The significance of time savings, however, must be interpreted cautiously 
since often- these savings are not only associated with the introduction of 
CAI or CMI but also with simultaneous revisions in course content. The 
primary savings in time seems to be associated with conversion of the 
course from a CI format to an II formally the addition of computer support 
(either CAl or,CMI)Ato II does. not seem^to increase the time savings signif- 
icantly (5 percent TOr CAI in five courses; 0 percent for CMI over seven . 
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cOMrses). No s.tudies were found which compaVed .CAI and CMI course completion 
times. The data with regard to th*. relationship between academic attrition / 
and computer-based instruction .ar^equi vocal . . Ttie Air Force reported an 

. Ificrease in attrition for four, courses on the Automate'd Instructiorrar 
System (AIS); however, iittrition iVi all courses at Lowry Air Force Base 

-increased during, the sam^ time period and the reasons ^or the increase were' 
uncectain. The Navy .reported an increase in attrition for -six CMI courses 

f over a 15-month peri6d and a decrease in at;trition for one course. The 
Army reported that academic attrition was about" th^ same for two couKes in 
basic electronics whethen^aught by CAI or CI. Another Amw study reported 
^22 percent lower attrition for a CAI course (Orlansky and String. 1979). 

^nu,. J^rJI^^ ^Lf^'^\^"?r that stjidents usually were favorable 

toward CAI or CMI relative to CI; On $h5^otRer hand, instructors were 
more favorable toward CI than toward CAI o^ CMI. 

In summary. CAI^ndjCMl are r^^8«Wto be as effective as CI in 
TqIqJ^^V^^ "^"^ when measured in terms of achievement^ (Orlansky and* String. 
1979), A more appropriate measure of effectiveness is the relationship of 

i?^. u^°?.Pr^°'^Tf® operational urvH^.. While the correlation is 
thought to be high, thisl^ias not been demonstrated Bither for CI or computer 
based instruction. A summary of findings on CAI and Emi when compared to 
CI is present«5^ in table 1. ^ 

f.-mo ll ^^^^^^^^ ^^^^ transforming a course- from CI to II saves student 
time in three way? First, higher aptitude students are pennitted to progress 
at rates consistent with their skill. Second, when courses are modified 
irrelevant materials tend to be eliminated. Third, special remedial materials 
can be provided to students on the basis of information gained through 
frequent diagnostic testing. uiruuyn. 

• The addition 6f computer support to II does not appreciably increase 
the amount of student time saved but may bring certain benefits such as 
•reducing costs for maintaining ,records and producing management reports 
Knlmc'^'^ instruction has ng direct educational effect on the student; 
the benefits are in the area^;^rse management. There is insufficient 
evidence at this time to deij^ine/ the exact nature and extent of the " 
sayings due to the use of 6ll. j/dividuals contacted during the study 

J • speed vjUhj^^h performance feedback is given and the 
availability- of increasetHnama?)ement information makes CMI worthwhile. 

Unfortunately, littje has been done to compare the cost effectiveness 
of various alternative instructional systems within DOO. For exaSp e 
after an exhaustive search for analyses dealing with CMI and CAI Jvstems 
Orlansky c^id String (1979) concluded that no data are ava?Table tSarpemit 
^hT^lnSfnn'rrn cornputer-based and conventional ?nstrucT on. 

This- finding i5 supported by data reported in subcommittee hearings .in the 

^i?;^ r^/ /'^''^V ^'^^ Learning Society! October 

1977). Efforts duriVg the TAEG study to find past cost comparisons of 
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TABU \. SUMM)fkY OF FI41DINGS ON CAl AND CM4 , COMPARED 
TO CONVENTIONAL INSTRUCTION 



\ - 


(Comptrid to ConvMitional Inttnidkxi) 


Cotnmenti 


CAI 


. CMi 


StudMl AchipvMMiit 


.4^ : 

Sum Of inoft 

♦ 


SanM 


• 

Parformanca measured only et achopL . 
Rebtion between performance at $dMt 
and on the |ob not demonstrated. 
Dbservad differencae not of practical 

importance, y 


CCUTM UmpMNNI 

Tmm 

• 


Na of 


.40 


— --s- 
8 


CMk Most tinw Mvingi maintained 
or increatod with extended iiae. 


Tim* uvad 
(MmNm) 


29% 

« 


44% 


1 

« 


Rsnpo 


•31 to 89% 


12 to 68% 


Naof 
Compsntons 


5 


7 


Computer-Mipport eaves Kttlo time beyond 
that of individuaired inatniction. 


TifM stvwl 
^ IniKvidiMi^ 
farad In- 
tiniction. 

CAI 


64% 

/' 

69% 


51% 

cmi 51% 


Student Attrition 


About tha saiM 


Slight bicrease 
may occur 


'CAi: venilimitod dau 
CMi: possible decline in student qualit) 


StudMri AttHudM 


Favorabto 


Favorable 




Imtnictof AttHudM 


Unfavorabto 


. Unfavorable 


' Very fimited date. 
Uttie attention given to instnictors. 


Cost 


Less. dM to 

student time savings ( 


Last, due to slu-* 

ent time savings ^ incomplete. 








Not known because cost data are limited 
and mcompletO) ^ 
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SOURCE: Orlanslcy and String (1979) 



• TAEG Report No. 7a 



computer leased and conventional instruction were equally disappointing. 

Studies have been performed which claim that one system is more 
efficient than another. However, thesie'^ail to provide conclusive proof 
because they (1) do not count the capplete costs of the system and (2) fail 
to show .the costs of alternative systems. For examj)le. studies of the Navy 
CMI system {Carson. Graham. Harding. Johnsorr. Mayo, ^ind Salop. *1975; and 
Hanson. Ross. Bowman, and Thurmond, 1975) failed to include a comparison of 
CMI costs with. the costs of alternative instruction systems, computer hard- 
ware co^ts. or, both. In addition, estimates of course development costs in 
one of these studies were, so low ai to be consi^iered immediately suspect. 
Representative data are, summarized in table 2. j!. j 

The following paragraphs- summarize the status of H in the Army. Air 
Force, and. Navy. The comprehensiveness of these status reports was limited 
by the t^rief time available' for the study. \ 

U.S. ARMY . ' . ■ 



The Army Training and Doctrine Command (TRADOC) is the single agency * 
responsible for all training and performance testing. The Army's training 
objectives have been described by Brown. Brandin. Cole. Marshall. Rubin & 
Waksman (1973) as follows: ' 

Training will be based on performance of students ("hands-on") as 
opposed to an instructor demonstration course. 

Emphasis of training will be more on functional context rather 
than subject matter. 

Absolute criteria rather than normative criteria will be used,. 

„ Testing will be performance oriented and measurement will be on a 
go/no-go basjs. 

Individualized instruction will be use.d to the greatest extent 
possible. • • 

Feedback will be provided at the training site and to training 
management.. 



A quality control system will be used 



An 



aspect of training emphasized by TRADOC is the use of Skill Quali- 
fication Tests (SQT) for advancement by proficiency as well as providing 
feedback to schools^on field performance of personnel. TRADOC began placing 
a heavier emphasis on II in the mid-1970s. The Army Training DevelopSient 
Institute (TDI). a TRADOC activity.' maintains^ the position that II (based 
on Systems Developed Instruction) incorporates the following factors- 
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TABLE 2. SUMMARY OF STUDIES REPORTING COST SAVINGS .ASSOCIATED* 
WITH* VARIOUS METHODS OF INSTRUCTION 
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critical job tasks • * * 
. performance based instruction 

performance based criterion testing / ' 

delivery system 
individual p*gi4l.^ ^ . 
r^forcement/performarirce warranty. 

Art accurate count o^ Army courses that are individualized is unavailable! 
Only a few of the Army's* elf -paced courses are open-entry and only about 
50 to 75 are open-exit courses^" TDI is currerttly assessing the extent of 
II in the Army (including how many courses are self-paced and how many are 
contractor developed or in-house developed) and determining problems and 
effectiveness of its use in the school. However, it appears that there is 
a trend to return to'Gl because of phanges in managers. •< 

present, the major operational CAI system inkhe Army is the Computer- 
ized Training System (CTS) formerly located at Ft. Monroe. This is the , 
original Army CAI system. The CTS is now located at the Signal'School , Ft. 
Gordon, Georgia. It teaches only 10 percent of three courses (radio, 
teletype, and avionijrs equipment repaiV) due to the hands-on nature of the 
courses. The CTS has 96 terminals and the combined load is 500 trainees. 

The Signal School is in the process of reconfiguring the CT^ to an 
Automated. Training Management System (ATRMS). Four hundred twenty-five 
trainees currently receive training on the system. The reconfiguration is 
being implemented from the beginning of the course; 800 trainees in the 
()asic radio-telegraph and teletypewriter operator courses who ^re in the 
last weeks of training are still in lockstep mode. Full operation is 
expected within 2 months. At that time, there will be approximately. 1 ,500 
trainees average ,oi:| board (AOB) in trkining. Plans are also being developed 
to put, the faculty development coursefe on ATRMS. This will give them a 
permanent record of all faculty improvement efforts. Six minicomputers are 
used to service 32 terminals for three basic courses. ATRMS has the capa- 
bility of managing- any trainee in more than one course at a time. Trainees can 
take the operator courses and at the same time take a course in Interwational 
Morse Code. 

The Signal School staff has encountered some difficulties in implementa- 
tion. Control of students has been ^a 'p'^oblem because of very large throughput' 

"vcourses. Further, courses depend heavily upon the written word as a result 
of direction to make training packages "exportable." The trend within 
TRADOC presently is to export as much training into the field as possible 

• and since CAI is not practical for such use, reliance has been placed on 
written materials. 

In addition to ATRMS there are a number of new CAI systems being 
implemented in the Army. Ft. Gordon will be the testbed for Adaptive 
Computer Training System (ACTS) and Reactive Electronic Simulator (REESE). 
The Army also plans to use the Educational Comfl^iter Corporation's EC-2/3 
systems for the following applications: 
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. / .Cobra H^li^Cpter training: 72 student stations for seven 
systems at Ft. Eustis , 

/ . • 

Black Hawk K^slicopter Againing: 22 student stations at Ft. 4 
Gordon, pi us . one. station for the composite system at Ft^ Rucker 

. ^ M60 Tank: 25 student stations a» Ft. Knqx and Aberdeen Proving ' 
Ground, i . ^^^ 

* 

' . M109/M110 Howitzer: four student- stations at Ft. Sill. 

Finally, Plato is being used by the Anny in the following trairting: 

Ft. Shafter: three terminals to deliver g'oneral education to 
military personnel 

Schofield Barracks:' five terminals for wheeled vehicle mainte- 
nance and to deliver general educatjon ta military personnel 

Tobyhanna Army Depot and Letterkenney Army Depot: terminals to 
deliver general education to tivilians. 

The major prototype of the Army CM I system, the Advanced Instructional 
Management. System (AIMS), is located at Ft. Sill. This system was adapted 
from the Navy's Versatile Training System (VTS) and is used as a personnel 
tracking system as well as a training management system. It is anticipated 
that AIMS will be art line in 1981 and wfll expand to include a rCmber of 
TRADOC activities. It is also anticipated that the Signal School's ATRMS 
will be subsumed under AIMS as they will be providing similar services. 

The TDI has been heavily involved in contracting related to II programs. 
TDI also has .a Styear program to investigate specific applications of CAI 
to Advanced Individual Training (AIT) for Military Occupational Specialities^ 
(MOS). They will select MOS courses having unique training problems in 
which CAI can be utilized as a solution and are Ja make extensive use of 
the latest microprocessor technology and develop unique low cost, cost- 
effective delivery systems. Twelve courses have been completed under 
contract ($1,434,400) and nine courses are being developed ($1,581,300). 
Further, a "modest" program costing $1,810,000 was initiated in 1976 to 
assist schools in systems development of instruction by providing contract 
resources. . ' • 

Finally, TDI has a cdntract with Appl i-Mation , Inc., Orlando, on 
Computer Assisted Instruction Training Delivery System (CAITDS) to investigate 
the use of tactical computers to. deliver training. It has four tasks: (1) 
' i<Jentify training applications for the Tactical Coftipttter Terminal CTCT), • 
(2) provide cost analysis and operational computer donditions and deployment 
of the TCT, (3) identify training applications for^th^ field (processor 
controlled system), and (4) -provide cost analysis and operational conditions 
and depldymerit of other tactical processors. ^ > 
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U/S. AIR FORCE . » 

There is no official Afr Force policy on I I (Goldman, 1979). However, 
according to Brown/T et al . (1973), the Air Force has the following goals 
and tra'ining objectives: *' - ' - / 

increase job-relevancy of trairvjng - 
Veduce training time 

• * 

use audiovisual devices creatively 

make instructors problem solving managers 8f instruction 

make training responsive to rapid changes in manpower 
requirements ' ■ \ 

tailor cburses to job performance objectives 

use cri terion-^$ed tests. 



DQspite the lack of official Air Force policy on II, there are pressures 
for self-pacing in advanced courses (the 7-skill level of the AF 3,' 5, and 
7-skill level system).. It is believed that II is of most value at this 
"Kevel of training because of heterogeneity of students in the courses. 

The Air Force has 110 self-paced courses, which according to Goldman 
(1979) are about 10 percent of Type II and III courses. The various types 
of Air Force courses are identified as: 



V 



Type I: 
Type 1 1 : 

Type *IH: 

V 

Type IV: 



Factory (contractor taught) . 

Special, modular, sJ*iort course, spedific equipment 
or new procedures ^ 

5 and 7 level, Basic and Advanced 'courses 
Field training detachments. ' 



It is important to note that in spite of the fact that only 10 percent 
of Type II and III courses are self-paced, 22 to 25 percent of the student 
load is in self-paced courses. ( . ' * 

The major A1r Force CAI system is the Advanced Instruction System 
(AIS). It was designed to teach four courses and is currently used to 
^teach one. Originally intended as an operational device it presently is' 
•being used by the research and development community. Currently, AIS is 
primarily used in a CMI mode (90. percent) for the one course resident in 
the systerti. The fact that" this system, failed to meet its x)riginal objective 
has been attributed to a lack of an effective program for institutionalizing 
the instructional innovation.^ In some cases, deliberate attempts to subvert 
the system were reported. Other CAl installations in the Air Force include: 

thirty^flato terminals at Sheppard Air Force Base for training' 
physician's assistants 
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twenty Plato terminskls at Chanute AFB^for special. vehicle maintenance 

Worldwide Military Command and Control System (WWMCCS) at Keesler 
AFB with 16 remote sites around the world 

several hundred 5-6 week courses iSsing' local computers administered 
.by the' Air ••Training Commanet (ATC) " , ^ 

. comnand- unique CAI/CMI; e.g., at Air University, Scott AFB, 
McDil} ARB , * 

base level computers for on-th^-job training in Civil Engineering, 
• personnel, and accounting/finance . 

personnel /administrative specialists are being trained in a basic 
3-level course usingXc^^^ processor with training tapes sent 

to local si tes. , 

There appears' to be an interest within the ATG in establishing "lessons 
learned" -in II, CAI, and CMI.- ^ / 

u / ■ 

U.S. NAVY 

Hi»torjcal aspects of the Navy's concern with II was provided in 
section I. The Navy clearly is in the forefront of the attempt to increase 
the efficiency of technical training through the use of 11.* For example, 
table 3 shows the projections for currently planned CMI installations (Van 
Matre, 1979). However, this inci^ased emphasis on II is only partially 
attributable to technological innovation and foresightedness . Table 4 
contains information compiled by the Master Chief Petty Officer of the 
Force (MCPOF) of the CNET. The table shows representative increases in\ 
technical time-to-train requirements for similar weapons platforms as a 
function^of time. When compared with projected training resource requirements 
{figure 1) using current methods. of training, it becomes obvious that either 
additional resources wiMl have to be obtained or training efficiencies will 
have to be effected if the Navy is to maintain the desired levels of readi- 
ness. In the present climate of austerity, it is un^-iKely that resources 
in the amounts required will be available. Thus, it seems likely that the 
use of II as one means of increasing instructional eff\ciencies will continue 
to grow. A 19716 report on the individual learning system at the Naval 
School of Photography, (NTTC Itr Code 01 Of 15^ October 1976) provides an 
example of the cost efficiencies possible. In a 4-year study, documented 
savings on course length reductions were 326K, 208K, and 255K,for FY 74, 
75, 76 respectively. In the same period, 737K in cost savings were realized 
from staff reductions. This efficiency was not obtained at the, expense of 
quality. Graduates of this program averaged 5 to 7 percent higher on the^. 
comprehensive course exam than under lockstep training and there was no 
apparent decVine In quality of graduates as perceived by fleet personnel. 

The current use of II In Navy training 1» known to be widespread. The 
following paragraphs describe the extent of the major components in the 
Navy including CMI, IMI, P.I, and CAI. *The comprehensiv^ess of these . 
descriptions' was limited by the time available for the study and the availa- 
bility of necessary data. The data in -table 5 provide a perspective 
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TABLES. PLANNED MILITARY CMI SYSTEMS 
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AIR FORCE 

AIS CTS 


ARMY 
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MARINE CORPS 
CBE 




Students Dally 16,000 


-2^400 365 


'> 

1,600 


2.000 




Courses/Schools 25 


4 . 4 
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. 4-8 CAi 
40 + tMI 
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Source: Van Matre (1979) 










*■ • . 
' TABLE 4., REPRESENTATIVE INCREASES IN TECHNICAL TRAINING 
TIME-TO-TRAIN REQUIREMENTS 




SURFACE^ 


DD-962 Class 
(196a) 


DLG-26 Class 
(1973) 


DD-963 Class 
(1973) 


"Sonar, Technicians 
(Man-weeks) 

Data System Technicians 
(Man-weeks) 


63 

« 

0 


504 

• 

0 

.1 


718 
500 




Machinery/El ectronlcs/Weapo 
• Technicians 
(Mart-weeks) 


ns 810 


3367 


4671 




AIR 

0 


F-8 Crusader 
(1955) 


F-4J Phantom 
(1966) 


F-14 Tomcat 
(1973) 


Total Maintenance Personnel 
^ (Min-weeks) 


573 


785 


1050 




* 

♦ 

SUBSURFACE 


SS-563 Class 
(1951) 
Diesel 


SSN-585 Class. 

(1959) 
Nuclear Attack 


SSBN-616 Class 

(1963) 
Nuclear Ballistic 


Total Technical Personnel 
(Man-weeks) 

'. i ^ 


1675 


4300 


6400 





Source: Master Chief Petty Officer of the Force (MCPOF). CNET 
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Figure 1. Future Traini.ng Resource Requirements Using 
^ Current Methods 
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Source: MCPOF, CNET 
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table's. ■ ENROLLS AND AOB" DATA BY METHOD OF INSTRUCTION' FOR FY 78* 



Method of Instruction 

Combination of Methods 
of Individualized 
Instruction 

Computer Managed 
Instruction 

Group-Paced or Lock- 
Step Instruction 

Self-Paced Instructor 



FY^78 
Enrol 1 s 

43.705 



22,867 
615, 75*1 
58,704 



TOTALS 741,027 



AOB 
Under 
Instruction 

5,116 



1,679 
42,970 



2,990 



52,755 



AOB 
Awaiting 
Instruction 

792 



4T9 



AOB 
Awaiting 
Transfer 

206 



26 



X 

3,407 ]i,838 



300 



296 



4,918 



2,366 



♦Specialized Training only (I.e., A,C, and F type courses) 
Source: CNET, Code N-302 
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by which to evaluate the Information In subsequent paragraphs of this 
section. The table summarizes enrollments and AOB as a function of tvoe of ' 
Instruction. •'^ 

COMPUTER MANAGED INSTRUCTION.' CMI Is .the largest component of what is 

?u"^^Iw?llLr^^^'^''^^ ^^^^ ^" NAVEDTRACOM. .With some activities of 
the NAVEDTRACQM. and certainly among user activities. CMI has come to be 
synonomous With II. self-paced Instruction, and. occas1t)nally. ISO. Unfor- 
•tunately, this confusion has often resulted in inappropriate Criticism of 
CMI. However, because of its Inherent record keeping capability it'is 
possible to provide some relatively detaileJ Information on the status of 
this system. 

The following data provides a thumbnail sketch of CMI system operation. 

Daily average student load (AOB): 6,7 95 (FY 78 ) 
No. locations using CMI: 5 . ' , 

No. schools currently using CMI: 14' 
Annual" throughput of courses on ,CMTT 66,57^ (FY 78) 
No. daily transactions on the computer: 26.508 (20 Aug 1979) 
. . Annual budget for computer operations: $3,550.000 (FY 78) ^ 

Table 6 identifies the major technical training courses currently on the 
CMI system and provides their AOB and annual > throughput. AOB and annual 
throughput for Individual course data processing (CDP) numt)ers of all CMl' 
courses are included in appendix C. 



TABLE 6. LISTING 


OF II COURSES CURRENTLY 


MANAGED BY 


COMPUTER - FY 79 


Course Name 


Type of Course 


■■ 

AOB 


Annual Throughput 


Basic Electricity and 


\ 






Electronic^s 


AP. 


4,506 


19,788 


Propulsion Engineering 


AP . 


1,177 


9,059 


Radioman 


Al • 


842 


5,223 


Aviation Mechanic 


Al 


386 


2,908 


Aviation Fundamentals 


AP 


1,099 


17,632 


Avionics Technician 


Al 


1,247 


2,968 



hw rirr Pl^^nts the interim FY 79 CMI Implementation Plan as promulgated 
by the LNET. This guidance may be superseded as NAVEDTRACOM real iqnment/ 
reorganization plans become effective. Nevertheless, the expansion of the 
CMI system is expected ,to reach a capacity of about 16,000 students at 25 
schools In six locations by the mid-1980s. . 
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TABLE 7. FY-79.CMI IMPLEMENTATION PLAN 



, ^Course 


— — ^^...^ ♦ — ^ 

Location 




Operatfonal Planning Date* 


IC 


San Diego, CA 




79/2 


PE EXP (BT) 


^ Great Lakes, IL 




79/2 


PE EXP (MM) 


■ Great Lakes. IL 




79/3 ' 


SK 


Meridian, MI 




■si I ^ 

ifff 


AK 


Meridian, MI 






FM 


^ureaL LaKeS , IL 




70/ il 


DP 


San Diego, CA 




80/1 


IT 


San Diego, CA 




80/3 / 


IT 


Great Lakes, IL 




'80/3 


IT 


Memphis, TN ,^ 


A- 


80/3 



*Expressed by FY/quarter; date at which.valldatlon revision of course 
develo'pment has been completed. ' 
**The 'advisability of Implementing SK/AK courses Is being reviewed because 
of staffing cuts. 



Source: CNET Itr Code 526, 18 Jan 1979 

4 

INSTRUCTOR MANAGED INSTRUCTION (IMI). The Implementation of II as an 
Instructional strategy has not been limited to courses m^anaged by computer. 
Anumber of courses, or segments of courses, employ an Instructor-managed 
"self-paced" delivery mode to transmit Instruction. Because of low throughputs, 
or other considerations, it was determined that It was not feasible and/or 
leconomlc to manage these courses by computer. In IMI, Instructors prescribe 
'instructional sequences, give and score examinations, maintain records, and 
provide counseling and guidance. Actual instruction, however, remains the 
responsibility of the student working with some form of II. 

The current reporting system makes it' difficult to identify all courses 
In the NAVEDTRACOM that can be classified as IMI courses. However, based 
on information av,ail able' from the N^avy Integrated Training Resources and ' 
Administrative System (NITRAS), it is possible to identify a number of 
specific courses that fall into this category. Appendix C lists the FY79 
enrollment, rtumber of graduates, and AOB for all technical -training courses 
identified as containing some degree of II including those managed via IMI. 
Major individualized courses which employ IMI, their AOB and throughput 
are 1 isted in table 8. s 
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TABLE 8. 


LISTING OF MAJOR INDIVIDUALIZED 
MANAGED BY INSTRUCTORS - FY 79 


COURSES 

C ' ' - — , 


Course Name 


Type of Course 


AOB 


Annual Throughput 


Yeoman "A" Course 


' Ai ;• ' . 


212 


1,146 


Personnelman "A" Course 


Al 


129 


893 


Supply Technician. "A" 
Course 


. Al ■ 


120 




Engineman "A" 


Al 


124 

V 


1,611 


Machinist Mate "A" 


(600 and 1200 PSI) 


638 


• * 

4,808 


Aviation Boatswain 
Mate, Fundamentals 


AP 


45 


1 ,028 



It is not possible to predict the future mix of courses with regard to 
instruction type with any degree of certainty. Hovyever, with a policy that 
maintains a preference for II (NAVEDTRA 106A, phase III, p. 124) and 
increased emphasis on efficiency in training, it is likely that II will" 
become ever> more imbedded in the NAVEDTRACOM. However, it will be. necessary 
to develop an algorithm for assignment of courses to IMI or CMI if full 
advantage is to be made of these management strategies. 

-4 1 

PROGRAMMED INSTRUCTION. Programmed instruction is norflially delivered via 
text;s' containing summary information, a narrative, or a sequenced learning 
program. It can; however, be delivered through a variety of njeai^ or combined 
with various management" strategi.es ; i.e^, CAI, CMI. V 

, Currently, programmed instruction ts a Ijey component in II. It is 
also the most difficult component to document in^terms of degree of use in 
NAVEDTRACOM courses. Programmed instruction is found in almost all course 
types, including CI where it 'might be used for remediation, the transmission 
of noncourse information, or augmenting of instruction; e.g., after hours 
study. 

The Chief of Naval Technical Training (CNTECHTRA) has produced a . 
Catalogue of Naval Technical- Training Publication's (CNTT-A-68) which lists 
some 800 -titles described in the catalogue as' being PI texts or part of a 
PI instructional package. These titles address subjects in a large variety 
of technical training topics used in many different courses. Lt is assumed 
that in, the time since this catalogue was published, other»-course materials 
have*been developed using PI. Jt should be noted, that other trypes of 
training; e.g.. Officer; General Military Training (GMT), probably employ 
PI but a survey of these areas was beyond the scope pf the present study. 
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It 1s anticipated that PI texts will continue to be a primary delivery ^ 
medium. The cost of audiovisual materials and the additional time that is 
required to develop and test other forms of instructional- deli very /manage- 
ment associated with II will limit their use in a time ef resource constraint. 
Barring technical breakthroughs which may alter this situation. PI .texts- 
will continue to be the most coninon form of II. ^ 

COMPUTER AIDED INSTRUCTION. The Implementation of computer-aided instruction 
in the NAVEDT^ACOM has been Velatively small comparftdvto other components 
of II. -- The cost of employing computers for the sole purpose of instruction 
has. until recently, been prohibitively expensive. In addition, the use of 
CAI has be^n' validated for only a few. very specialized types of training. 
In the short t^me available for the study, a surprising number of CAI 
programs were identified. These are listed io table 9. While not exhaustive, 
this infbrmat ion' gives an impression of rapid growth In this aspect of lU 
for technical training. 

It is dimeult to identify all CAI delivery systems in the Navy 
because of maifagement practices associated with procurement. Major systems 
are reported and carefully, tracked within the NAVEDTRiltCOM . However, desk 
top calculators aad training devices employing computers are procured\and 
managed separately from large systems and are identifiable only through 
tediou,s examination of records arrd Interpersonal contacts. A management 
information system which permits tracking of all CAI systems will soon be ^a 
requirement if duplication of effort is to be avoided and effective management 
control exercised. 

It is likely that experimentation with CAI delivery systems, particularly 
for specialized types of training, will continue. However, it is unlikely 
that there will be widespread expansion of CAI or repTacefment of CI or CMI 
until software costs can be lowered significantly, i^t a time when hardware 
costs are dropping precipitously, software costs continue to be the limiting 
factor in any computer-based development. , 

It should be noted that the information contained in this section was 
obtained in a 4-month period. The data ^hould be considered representative 
as an exhaustive survey was not possible. Similar analysis in other areas 
of training and education are required to have a complete picture of the 
extent of II in the NAVEDTRACOM. 
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TABLE 9. .COMPUTER-AIDED INSTRUCTIONAL PROGRAMS PLANNED/IN PLACE IN NAVY TRAINING 




CAl Identification 


1 rfli ni ng usv 


Number of Students 
Stations 


Locations 


T 

R^rks 




TICCIT (MITRE/Hazeltlne) 


. S-3A 


Undetermined 


North Island 
Cecil Field 


In Use 


ro 


Computer Corporation's 
2- and 3«-dlMnsiona1 
panels with CRT) 

GETS (General Electric, 
computerlzedi self-* 
contalrledi Interactive 
^ training console) 


AE 
AS 

Marine 
Marine 

nAvair 

TRIDENT. (stratealc 
Weapons Training) 

TRIDENT (Engineering. 
Operations Training)* 


91 

103 
100 
60 

Undetermined 

15 
12 


Memphis 
Memphis 

Camp Pendleton 
Twentynlne P^^ms 
Undetermined 

/ 

Bangor 
Bangor 


J 4 systems under 

contract 
8 systems 
27 systems 

200 additional stations 




Device 20S17 


' - OS "A" School 




Undetermined 


NTDS Training 

FY 84 Implementation 




CT(0) CAf System 
« 


CT "A" School 


. 15 


Undetermined 


Basic Communlcatloni and 
CT message handl Ing ^ 




Not Designated 


F-18 


Undetermined 


Undetermined 


Planned for Future 


f 


eevice lOHl ^ 


EW Operations 
and Maintenance 


60-70 


Corry Field 


300 Learning Carrlls 
will be under CHI 














\ 
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^ SECTION III . * 

FACTORS. AFFECTING THE EFFICIENCY/EFFECTIVENESS OF 
INDIVIDUALIZED INSTRUCTION / 

• This section identifie's ^^<\ discusses factors w^iich influence the effecr 
tive development, delivery, and management of instruction. The analysis 
focuses only on those factors which cm effect changes toward imprq^ing the 
efficiency/effectiveness of II in the NAVtDTRACOM. The present Navy 'I I 
program is a massive undertaking. Although its full extent is difficult to 
document, its CMI component is the largest aj[Jd most successful system of 1tS- 
kind in terms of numbers of students processed and'numbers of courses resident 
in the system. It should' also be noted that the Navy has consistently been 
in the forefront of the development arid implementation of computer based 
instruction. Because of its Visibility, the Nav^^" program has received more 
than its fair share of- criticism. 

V It has been demonstrated that II is as effective an instructional 
strategy as CI. Therefore, there are only two major relevaift-,consi derations 
for the evaluation of II: cost effectiveness and management effectiveness. 
The remainder of this section provides information bearing on these issues. 
Specifically, factors are identified which Singly or in combination mf 
impact on the cost or management effectiveness of II > Each factor is then ' 
discussed and specific illustrations of its effect on instruction are pro- 
vided. The factors are arbitrarily grouped into those dealing with: 

• . organizational structure far II ^ ' 

attitudes , ' ^ 

. . resources 
data bases 
^ . - instructor/manager training 
course administration. 

ORGANIZATIOr^AL STRUCTURE FOR INDIVHDUALIZED INSTRUCTION 

A primary factor in the efficiency of II is the organizational structure 
which has evolved to support its implementation in NAVEDTRACOM. An appro- 
priate organizational structure is a necessary but insufficient condition to 
the effectiveness of an instructional system. Seidel and Wagner (in press) 
have suggested that desirable organizational characteristics associated 
with a complex innovation such as the large scale implementation/ of II 
include: . ' " ^ 

a clear line of project' contrbl with congruent allocation of 
authority and responsibility 

frequent communication for monitoring expectations and under- 
standing 

continuous communications mediated by the project manager. 
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Focus on the$e characteristics Is panjcDlarly relevant to the analysis of 
the Navy organizational structure for II. The <:urrent command structure of 
NAVEDTRACOM was developed t©v support an instructional system whi.ch was and is 
characterized by the use of platforti) instruction and school house (decentral- 
ized) development and management of instruction. The implementatibn of II, 
the move toward centralized development* of instruction, and increased central- 
ized management of instruction via the computer^ have placed additional emphasis 
on the need for integration and coordination Across functional lines. 



Figure 2 shows command and management relationships, CMI project, manage- 
ment, and technical guidance relatipnships for CMI, a major deMv^ry systfem 
for II. It is used here as an example of the complexity of organizational 
relationships wh'ich have developed to support this Inajor facet of II. In 
additipn to the various relationships depicted, each of the CNET Assistant 
Chiefs of Staff (ACQS) has responsibility foY policy guidance in his respective 
area. Analysis of the structure depicted in figure 2 indicates the lack of a 
clear and unambiguous line of control and a potential for disruptions in 
communication and lapses in coordination. 

During interviews conducted as a part of the' present study, organiza- 
tional structure was identified more-frequently than any other factor as 
having a significant influence on the effectiveness of II. A major difficulty 
resulting from the current structure is that of establishing accountability 
for specific aspects of II. This difficulty was perceived to exist through- 
out NAVEDTRACOM except fpr ADP organizational units. Specific areas in which . 
this difficulty was man1fested>^and which are discussed in this repor*t were: 

. r 

ambiguous policies for the selection of instructional strategies.* 

lack ofystandard policy for the use of course administration data 
availame fr*om the CMI ^ystem 

• . lack of coordination among activities responsible for centralized, 
course development and resource allocation/acquis.ition 

inability to respond to requests for quantitative cost effectiveness/ 
. , evaluation data 

« ■ ■ . * 

Lapses in responsibility' and/or communication. 

^ In addit4en, the lacic of a single office/activity with responsibility' 
for the integration and coordination of all aspects of II has contributed 
significantly to these problems. What now exists at CNET are offices with 
individual concerns for policy in instructional development and implementation, 
operational management of type traijiing (technical, special, air), ADP support, 
or the management of centralized instructional development with regard to II. 
This decentralization of structure is glso maintained at lower echelons of 
the command. This sVudture has promoted compe,tftion for resources, ill- 
defined boundaries of responsibility, and an occasional inability to respond 
complj^ely and effectively to user concerns. ; 
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ATTITUDES* ' 

Currently, a slgnlflcjgit number of Navy personnel have the attitude that 
II is ineffective and/or inefficient. In most instances, thi^ negative 
attitude it not justified. Rather than focusing on specific problems such as 
the quality of inrstructional material, shortage of resources, or conflicts in 
policy, criticisms are usually nonspecific in focus. 4 

Negjative attitudes and resistance to change are likely to result from a 
lack of information, a failure to involve those activities required to imple- 
ment the new program, and/or a lack of high level coirmitment to 'that program. 
Unfortunately, these effects tend to be generalized to all aspects/elements 
of the system rather than restricted to the specific components that may be 
ineffective. 

•k 

This generalization of attitude often results from a lack of knowledge 
of the management structure or the failure to i^wterstand that many educa- 
tional or instructional decisions are dictated" external constraints. In 
some instances there may be a perceived failuce of -the system to respond to 
expectations or requirements. S 

The discussions held with personnel irwolved in II indicate mixed per- 
ceptions of II. Some product users appear to be generally dissatisfied with 
the capabilities of graduates of II programs. Managers within the system, 
however, tend to view II as a satisfactory means of meeting increasing instruc 
tional requirements. Student and instructor attitudes range from total 
support to disenchantment. There isr a small coterie vehemently opposed to II 
who are convinced that CI is the only form 'of instruction. However, the 
trend appears to be toward a more positive view of II as more experience 
with this instructional strategy is acquired. > 

Differences in attitude appear to reflect the degree of involvement with 
II. Moreover, many of the negative attitudes apparently reflect deeply held 
beliefs about the' value of CI or an incomplete knowledge of what II is and, 
how it opjgraters. The essential'*pdint is that, regardless of cause, these 
perceptions do exist and must be addressed. Equally important is the identi- 
fication of the conditions that can be addressed to ease the problem. 

A number of factors were identified during interviews which appear to 
have contributed to the development of negative attitudes toward II. These 
are identified and briefly discussed below: 

Confusion in terminology.% As indicated earlier,' this i^ a per- 
vasive factor^ Clarify of terminology is essential to the estab- 
lishment and communication of concepts, policy, and operations. 
The Incorrect use of specific terras has led to misunderstandings 
and inj^^ppropriate criticism of the entire system. 
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Changing roles of instructors . The changb from platform instructor 
■ to learning center supervisor (ICS) and from ICS to learning 
center instructor (LCI) fcwis. required considerable alteration to the 
functions of instructional personnel. Failure to recognize the 
functions of this new role, or a lack of training to prepare indi- . 
viduals to perform these functions, has been reflected in disaffec- ' 
tions with the system. 

Changing roles for students . The implementation of II h?s also 
changed the student's role. These changes have resulted from the 
requirement for more independent work, interaction with computers, 
and adaptation to different instructional environments. Combined 
with the changing quality of accessions and difficulties in iden- 
tifying, and providing appropriate incentives, these factors have 
affected student attitudes. It appears, however, that students can 
adapt more readily to II if it is encountered early in .the training 
pipeline. Shifts between II and conventional methodologies may 
also result in student disdrientation unless adequate preparation 
for these shifts is provided. 

Impact of external constraints . Personnel who seldom come in direct 
contact with the development, delivery; and management of instruc- 
tion generally do not have an appreciation of the constraints which 
operate on the system. Often, fiscal and management considerations- 
are imposed which result in a less than optimal instructional 
program. Unfortunately, criticism is then leveled at the instruc- 
tional program itself rather than at the constraints which have 
affected i t. , 

Cownunication failures . This element is basic to the presence of 
negative attitudes about II and has also been discussed in the 
context of organizational structure. It is mentioned again in order 
to emphasize its importance to the overall effectiveness of II. 

Changes "Nn course content . Concurrent with the initiation of II, 
af{6/or~t}^ introduction of CMI, resource constraints have forced 
instructional managers to^review what can reasonably be accomplished 
in training programs at ail levels and to adjust programs accord- 
ingly. This sometimes results in graduates with different qualifi- 
cations than previously produced. User failure to recognize that 
changes have occurred in instructional strategy and course content, 
with attendant implications for training responsibilities by on- 
the-job training (OJT), has resulted in some unwarranted criticism 
of instruction. 

Measurement of performance . ^ The present lack of data on the job 
performance and/or retention of knowledges and skillsjof school,, 
graduates has made it difficult to assess the validity of user 
criticisms of instructional programs. 
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UndefUandlnq of instructibhalfitechnoloples . Much negative feeling 
toward II results from the perception that 'depersonal Izatlon of 
Instruction and <:hanges In quality of Instructional material 
.result from the use of the computer to manage instruction. /Neither 
, of these perceptions is accurate, but they do show a lack /f under- 
standing of the parlous, aspects of II. Further, the requ^ements 
of computer based management for stringently specific InfOtatlon 
' • and relatively rigjd operating guidelines may contribute to these . 
perceptions since they might appear to be dictating Instructional 

RESOURCES • ■ • 

Althotjgh training effectiveness and efficiency have already been dis- 
cussed. It should be reemphaslzed tha< decisions made about Instructional 
systems or programs always reflect a balance between these two concerns. 
Since research has shown that there is little or no difference in training 
effectiveness between conventional and individualized instructional strategies 
the resource efficiencies that Impact may be the most crucial elenfent in the 
evaluation of II. Twq points must be considered in the assessment of the 
efficiency of II: 

To date, research has concentrated on an expression of efficiency 
in terms of a savings of student time to complete specified material 
The use of this single criterion as a benchmark may fail to take 
into account concomitant changes in ^iirriculum material. Thus, 
available estimates of efficiencies may be confounded. 

. There exist only a few relevant cost data bases for use in 

comparing the various economic elements'of instructional programs; 
e.g., the CNET Resource Management System. Until more complete " 
data resources are developed and mewintained it will be difficylt, 
if not Impossible, to provide unambiguous estimates of cost effi- 
ciencies. 

In-' addition to the purely economic basis for choosing an instructional 
strategy, there are several compelling reasons why more cOst efficient 
training must be sought. For example, information presented in the previous 
section indicated a trend toward increasing complexity of training require- 
ments with a concomitarft requirement for Increased training times. The 
combination of these trends with a requirement that training resources remain 
proportionately constant clearly establishes the need to Identify more effi- 
cient means of training. Individualized instructions is one possibility for 
effecting these efficiency measures. 

The decline in availability of tr^Mning resources >n general will 
continue to have an impact on specific tiQaining^ograms.. Thfe gerteral reduc- 
tion requires the reprogramming of availaBle^sources according to shifting 
or changing priorities. This often results in the development of adversary 
roles among training programs, courses, and systems as they compete for these 
resources. The existence of this competition implies that those units that 
c^n justify expenditures most effectively, while at the same time receiving 

« 
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active support from policy making personnel, will iru:ur the least negative 
Impact on their* programs. As has been discussed previously, an attitude of 
t(rtal commitment for II programs Is not currently perceived to exist. The 
data bases from which justification evidence'could be compiled is fragmented 
an* Incomplete. . * 

The appropriate method of assessing the efficiency of II is through life 
cycle costing (^ee appendix B). Unfortunately, li1<e any high cost invest- 
ment, ^ese life cycle estimates are subject to events which may affect the 
accuracy of the initial estimates; e.g., t^e introduction of new technology. , 
The failure to apply an appropriate costing model or to take into account 
external events operating on the evaluation may result in inappropri,3te or 
inaccurate data. 

Another factor which affects the relationship between resources and 11 
is the influence of external requirements or decisions on instructional 
issVies. For example, external requirements to justify particular expendi- 
tures or to employ a particular instructional strategy may preempt the follow- 
ing of prescribed I^ procedures. Thus, aptimum education/training programs 
may 6e made subservient to resource allocations. These external consider- 
atioijis also affect the strategies by which claims are made for resources. • 

' A final general issue which affects allocation of resources is grounded 
in the changing nature and qualifications of the student population to be 
trained. The decrease in the total base population from which recruits are 
taken, the increasing numbers of women being trained, and the lower entrance 
level skills of recruits will assume more and more significance in the alloca- 
tion of resources. Additionally, this set of circumstances may impact 
directly on selection of strategies or delivery systems and, thus, indirectly 
on resource use in operating these systems. 

Clearly, the issues are complex and interactive. They are cited here as 
a basis for the interpretation and evalua,tion of other more specific findings. 
Several specific observations related to resources followr 

1. ^ Increasing instructional requirements and decreasing resource 
availability have provided an impetus for tire continuing development of inno- 
vative and efficient instructional strategies and delivery systems. i 

2: There is apparently a satisfactory level of resource avai*! abi 1 ity 

for ADP support requirements, particularly in the hardware area. Capability 

for CMI expansion, for example, exists to levels that should acconiinodatfe 

requirements for at least the short term. 

* 

3. A primary result of the reduction in resources has been the reduction 
of ' suppoK services at the schoolhpuse level. This reduction in services. 
While detrimental to all programs, is particularly disruptive to those courses 
using^II as an instructional strategy. For example, in numerous instances 
instructors are forced to assume responsibility for support functions. For 
II courses, where the. original instructor complement was established on the 
basis of past estimates of student/instructor ratios and average periods of 
instruction, this added responsibil^ity may Increase the workload to critical, 
levels*. During peak loading period?. *the additional support requirement , may 
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necessitate cancellation of leave or other acfcfons that will affeqt the' 
morale of the Instructor/management staff and» ultimately Impact on. Instruc- 
tional quality. . . « 

The shifting of responsibilities and dutj-es among schoolhouse personnel 
makes It difficult to maintain an accurate breakout of^labor. Further, the 
disproportionate Impact of resource cutbacks on support activities may pre- 
cipitate a circumvention of the manpower accounting system in the belief that 
if such billets were identified as being filled by instructor personnel, the 
instructor billets would be perceived as being unnecessary and would eventually 
be taken away. This practice, unfortunately, has the direct negative impact 
of reducing^ the credibility of the manpower repbrts coming' out of the instruc- 
tional envfronment. >^ " 

4. SigrAf leant waste of resources is occurring as a result of an 
Inability to integrate the ISO process and the POM/budget cycle. In the 
Implementation of II. the lack of coordination between course development and, 
the POM cycles can be reflected in several specific ways. For example, there 
may be discrepanctes between course development and ADP hardware acquisition 
or between course facilitj^ requirenients and physical facility renovation 
budget insertions. The po'tential extent of this problem can be appreciated 
from a recent estimate that over $200K was expended on equipment acquired but 
not utilized at tme NAVEDTRACOM locations during an 18-month perjod. 

5. There is currently a widely held perception by lower echelon activ- 
ities that there isf a lack of management support for II programs. This 
perception is at l^t partially based on the low priorities assigned for 
resource support of those component systems that are associated with or 
supportive of II. This concern also extends to personnel support. 

DATA BASES 

I" 

There are three primary areas in which management information is required 
for the operation, management, and evaluation of any instructional system. 
These are course administration data, cost data, and training effectiveness 
information. Course administration data are inherent in the delivery of 
Instruction. They include planning and -status data at higher levels of 
management as well -as specifics of course administration and studept per- 
formance at the schoolhouse level. Cost data may be categorized into devel- 
opment costs and operational costs and include those costs associated with 
the direct support of the instructional program by other agencies; e.g.. 
computer support. Jrairring effectiveness information may reflect the degree 
to which a course has met its training objectives (internal evaluation) or it 
may reflect the degree to which course objectives are related to performance 
requirements in the Fleet (external evaluation). Various aspects of the 
above data bases can thus be combined to address questions of efficiency 
and/or effectiveness. The following paragraphs provide a brief overview of 
t;he current status of data bases available for the assessment of II. , 
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COURSE ADMINISTRATION DATA. There are considerable course, administration 
data available on ajl technical training courses in the various files of 
NITRAS/ While a great deal of detailed information is available through this 
system, it provides only a very broad categorization of courses in tecins of 

The CMI system maintains additional administrative data on those courses 
presently on the system to the level of individual student responses and 
provides for the management of a student's progress through the course. This 
system also provides a series of reports appropriate to the overall manage- 
ment of II at the school house. Additional management reports are being 
developed on a centralized and individual basis as needs are identified. It 
may be appropriate at this time to conduct a requirements analysis to preclud 
duplication of effort, to insure that appropriate management tools are avail(^ 
able, and to identify any additional training needs regarding the use of 
available information. 

No single data base was identified which permits either a complete 
reporting of status or the evaluation of computer based instruction in the 
Navy. Such systems have been identified (section II) but have been associ- 
ated with the research and development cycle of training device procurement 
.making it diffijcult to track and evaluate their operational use. 

COST DATA. Data which permit a comprehensive cost benefit analysis are not 
available. The CNET Resource Management System (RMS) can provide operational 
costs of courses, and the acquisition and operejtions cost of the hardware 
supporting CMI are available. However, unambiguous course development costs 
are not available^ Further, a meaningful comparison of II with CI ernd CMI 
with IMI in tenfis of efficiency will require specific cost data associated 
with instructional development' and course operation. ^ 

The only cost comparisons of M and CI have been in *terms of projected 
savings in student time. It has been found that considerable savings are 
obtained' through II. It was noted earlier that portions of thes^ savings may 
be attributed to changes in curriculum, thus confounding any generalizations 
about II efficiency. It has been suggested (Orlansky and String, 1975) that 
a 10 percent Increase, in efficiency is realized when computer management is 
added to a well designed individualized course. Clearly, quantitative 
response to inquiries regarding the efficiency of II are not possible at the 
present time without the development of appropriate data bases. 

TRAINING EFFECTIVENESS DATA. The training ^effectiveness of IT has generally 
been assessed by comparing end of course achievement scores of students 
in II and CI courses. It is general/ly held that II is at -least as effective 
as CI in those terms. In essence, this amounts to an internal evaluation or 
an assessment of the degree to which instructional strategies are equivalent 
in meeting course objectives. Such evaluations are being conducted by school 
on a continuing basis. 
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The development of a comprehensive/standardized data base by CNET for 
external evaluation is currently underway. However, there currently exists 
no data base which permits a direct assessment of the effectiveness of II on 
a command- wide basis. 

INSTRUCTOR/MANAGER TRAINING 

The implffmentatioo' of II in the NAVEDTRACOM has had a profound influence 
on the role of the instructor and manager in the learning process. In II, 
the pro|$Qjifeion of instructor effort devoted to teaching is less than that 
devoted to the roles 6f counselor, classroom manager, automatic data process- 
ing technician or master-at-arms. However, cgurse-related activities such as 
Instructional development or test and evaluation have changed. These changes 
dictate the need for effective preparatory training. in the unique aspects of 

I I m % 

t 

In discussing the impact that training of instructors and managers has 
on the effectiveness of II. two additional points should be borne in mind: 

The role of the instructor in II is still evolving and the best 
utilization procedures, optimal assignment policies, or the extent 
of various kinds of training that should be provided have not been 
established. Accordingly, it is important that provision be made / 
for investigation in these areas to continue. ' -/ 

Training in II should not be limited ^o instructors. There is* 
ample evidence that the requirement for training in the delivery • 
aspects of II extends' to all levels of manageiient. Those in^ilviduals . 
in administrative and/or managerial roles sbplild also be trained in 
the optimal application of II to the learning process. 

Dm^ing the course of this study effort, specific areas of concern related 
to the \ra Irving of Instructors/managers were identified. These problem areas 
are discussed below. x 

t 

1. Although recognized as important. litt>e in the way of tangible 
support has been provided instructor training for JI. This lack of support 
is reflected in low priorities assigned to instructor/manager training. 

2. There is little standardization among training activities in train- 
ing provided for instructors assigned to II progrartis. Some training activities 
have based their programs on the assumption that an II instructor should be 

as broadly trained as possible and require that all available training courses ' 
betaken. Other activities require only completion of the LCI course now 
offered at the various IT schools. In either instance. OJT may be provided. 
Neither of these approaches is optimal; however, one of these may be ineffi- 
cient and the other ineffective. 
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• 3. There is a widespread perception that the LCI course, as currently 
configured and administered, is of little value to potential instructors in 
the delivery of II. Trailing needs of .the LCI hawe.not been adequat^y 
identified. The course itself, intended as a 5-6lj\ Individualized course of 
instruction, normally takes only 3 days. This is no| considered sufficient 

« . . . ' 
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to provide the breadth of Information that is required for skilled peV-for- 
mance in a learning center. It is questionable that OJT can make up for 
these deficiencies. . • 

4. In al most all training activities* the effects of a continuous-'flnd 
sometimes s>^bstantial pressure- on the instructor to move students through the V 
program, using techniques with which he may not be total ly: f ami! iar or com-. • 
fortable, can be seen in the behaviors and attitudes of the instructor popula- 
tion.. If these conditions persist, and as time in job increases, anxiety, 
alienation, and/or boredom levels may increase. Several training activities 
control this effect by rotating instructors among ^nstructional and support 
functions. 

The essential point here is that this type of problem is being handled 
on an individual basis with little or no coordination among training activ- 
ities or even among schools at the same activUy. Despite these difficulties, 
individual schools seem to be meeting their own unioue instructor* training 
requirements through OJT. It is affparent that both the knowledge and the 
willingness to address training problems is present within the instructor 
community; however, the current piecemeal approach is not the most effi- 
cient way to, apply available skills and knowledges. 

COURSE ADMINISTl^ATION . 

The factors discussed in this subsection are related to various aspects of 
course administration. 

TESTING f'OLICIES. The total effectiveness of II is influenced by the compre- " 
hensiveness of the evaluation programs associated with it. Students presently 
enrolled in courses managed by CM! are limited to a multiple choice format, 
hence, restricting evaluation to the assessment of retention via recognition. 
Speci^cally, this policy has restricted flexibility in developing item 
alternatives in both progress testing and remediation testing. It has not 
generally been used to assess the recall of material, although this may be 
possible through tnT appl ication of ingenuity in the use of multiple choice 
formats. However, several instances were observed in which initiatives were 
being taken at the school house to provide for testing via recal'l using 
additional written and oral examinations. In addition, alternatives to the 
current hardware limitations associated with student input ^re being evaluated 
.as. a part of an overall systems analysis of delivery strategies. 

PREDICTED COMPLETION TIMES (PO). The purpose of PCTs in II is to provide 
guidelines for the assessment of individual student progress by allowing fOr 
monitoring of progress during the course and by establishing a student's 
class ranking. The use of PCT to influence the progress of a student is not 
completely consistent with an idealized model of II but is necessary and 
consistent with the requirement that the Navy train its personnel in the most 
effidient manner. Because it is a majffr basis for evaluating progress the 
average accuracy of predictions and the specific components in tiVe predictipn 
equation are critical considerations. Several respondents indicated that the 
level of accuracy in individual predictions of the PCT was unacceptable. ^ At 
least a portion of this inaccuracy was due to the fact that reading compre- 
hension was not included in the PCT equation until recently. Since the 
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courses are verbally loaded this discrepancy was to be.^pected. In addition, 
what appears to be slavish adherence on the part of some activities to the 
use of the PCT as a point estimate. Instead of an average which Includes a 
range of acceptable values, appears to h^ve complicated the proWerti. Further, 
the lack of uniform application of the PCT across activities makes diHtt 
comparison of various evaluation data difficult. , | 

COURSE LOADING.' Given the availability of rel a ted%upport services; e.g.. 
billeting, the momentary student capacity of anv course Is determined by the 
number of classroom seats or carrels. Thu^. b/th conventional and individual- 
ized courses suffer the same stresses associated wfth an Inability to level 
load. The perception that Individualized coufses would somehow act ta minimize 
the Impact of uneven loading has proven to Jje Inaccurate. The observable 
effects from aperiodic loading In IT Include: 

high student/Instructor ratios 

doij^)le shifting at peak loads i 

Increased time In queue 

disruption In support areas--mess1ng and billeting 

an increased sense of depersonalization 

decreases In motivation of students and. instructors 

Increased difficulties In tracking students. 

HARDWARE. For the most part. ADP equipment associated with the management of 
II in CMI Is very reliable and meets system specification^. Figure 3 shows* 
a summary. of a typical day's interaction with the systems F1gure'4 shows 
CMI central system availability over a selected time period. However, several 
considerations, with respect to hardware are appropriate. 



State-of-the-art technology Is rapidly progressing and elements of the 
present system are becoming obsolete. For example, alternatives to the 
present paper input system neec^ to be Identified and evaluated. The lack of 
flexIMlity in the system's capability to accept student responses has been 
addressed in a previous section. Decision guidelines for the centralization 
or decentralization of present and future system- configurations also need to 
be established. Initial steps are being taken by CNET to accomplish these 
requirements. 

X The capacity of the present system is more than adequate given the 
or1g,inal and modified Implementation schedule. However, changes to these 
schedules and the Increased demand for on-line interactive delivery of instruc- 
tion requires an in-depth analysis of the present and future potential of 
distributive, processing. 
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Figure 3. Typical Frequency of Trainee Interactions with CMI fof^ Nine 
Courses in Session on 20 August 1979 for 24-hour Period 
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The only difficulties associated with the present hardware system are in 
the transmission of information. This occurs primarily at a single location 
and is not considered a major prdbTem, It is anticipated that -the problem 
will be solved by advances in technology or decreases in cost of alternative 
transmission modes. 

MANAGEMENT INFORMATION SYSTEM REQUIREMENTS. T»\i?.study identified no manage- 
ment information systems capable of producing all information required for an 
analysis of all aspects of II. . There is a need to determine whether the 
current reports available through the CMI system represent an optimal response 
to the information requirements of the various agencies of the training 
community. There is little doubt that sufficient data is or can be made 
available. Figure 5 provides an indication of thfe variety of reports avail- 
able from /CMI and indicates levels to which they might be applicable. -How- 
ever, the^e reports do not reflect a requirement to integrate, for higher 
levels of/ management, information on all aspects of II in the technical 
training /environment. Also, the figure does not indicate the information 
requirements for the coordination of instructional, budgetary, and planning 
functions. 

COURSE ARTICULATION. A common difficulty which has also occurred with respect 
to the implementation of II is the development of courses without sufficient 
concern for integration with other courses in the training pipeline. Speclf- 
ica.lly, school personnel have Indicated that in many instances follow-on 
schools and fleet recipients have not recognized that (bourse content has been 
modified-and that the responsibility for aspects of instruction have shifted. 
This has produced unwarrantectvcriticlsm of the instruction being offered at 
the school . 

A problem which may be unique to the integration of II courses with con- 
ventional courses is the disorientation associated with the movement of 
students between these radical ly^ different instructional strategies. This 
disorientation may be reflected in both instructional and noninstructio'nal 
areas either of which will result in decreases in training efficiency/ 
effectiveness. 

The specific impact of shifts between 11 and CI on the planning for, and 
access to, schoolhouse resources is unknown at this time. However, it appears 
that thi5 is an area which has been ignored and which, if analyzed and con- 
trolled, may offer the potential- fbr significant cost savings and training 
efficiencies in the training pipeline. . 

STUDENT ADMJNIsfkATION 

This section describes the' impact of administrative factors specifically 
related to students. 

HOLDING TIME. There appears to be no consistent policy with regard to the- 
use of students during holding periods. The length of the holding period is 
directly related to the peak loading of courses. There is little information 
which permits an analysis of the direct impact of holding time on the effec- 
tiveness of II. The type of problem associated with holding time is related 
to where that time occurs in the pipeline. Students placed in a holding 
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.status prior to the<r initial technical tra^^ning have shown motivational and 
attitudinal problems. This effect may be compounded by the transition from a 
highly structured traioing environment (recruit) to the more permissive 
environment associated with II. Technical training student^ who are placed 
in a holding status between their initial technical training course and their 
follow-on training may also experience motivational and attitudinal diffi- 
culties but. in addition, may experience some deterioration of knowledge/ 
skills through disuse. 

Several commands have- developed Innovative/^pproathes to the use of 
holding time: . ^ 

use of advanced students as Instructors in remedial programs 
temporary assignment to fleet unit§ 
assignment to remedial programs 
usjB of time for GMT. • 

The capability to apply these innovative approaches is dependent upon location 
and 'environment. However, they are examples of what can be done to increase 
the effective use of holding time. . 

REMEDIATION. There are two Vorms of remediation which may impact on the 
effectiveness of traigiing. The first is Basic Skills remediation which most 
often occurs early in the training process. This form of remediation generally 
involves reading s^kills and, perhaps, basic mathematics; e.g., the Academic 
Remedial Training System (ARTS). Improving Your Navy ReadincL Skjm (Curry 
and Kincaid, 1979) identifies a goal of ARTS as raising the reading ability 
of recruits to a 6th grade level, the minimum. for adequate understanding o>f 
recruit training materials. This does not imply a capability to enable these 
people to understand more technical training materials. The second form of 
remediation may involve course related remediation which is more specific in 
content and directed toward the accomplishment of course objectives. It carl 
be expected that as. the quality of accessions decreases there will be require- 
ments for increases in both forms of remediation. Future policies dealing 
with entry level requirements, waivers, and length of programs rmjst take 
these factors into accounts [ ' \ 

• An issue related to remediation is the policy of accounting for student 
time when in a remediation status. • In some instances, aM student time is 
reflected in the accounting thus inflating time in course averages. In 
others, student remediation times are not a part of the course completion 
time thus lowering average course completion times. The lack of consistent 
policy makes comparability of course effifciencies difficult. ^ 

With regard to CMI, policies for the provision of remediation internal 
to specific courses are perceived to be somewhat inflexible and driven. by 
noneducatfonal factors. The content of remediation may be affected by the 
lack af resources to develop desirable remediation materials. *ihe program- 
ming requirements associated with the delivery of remediation on CMI may 
create conditions which promote a standardized system but at the same time 
these may not be suitable for all courses. 
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If recruitment and assignment policies continue to be established Inde- 
pendently of training resources and capability considerations, these problems 
will become more pronounced. * 

INCENTIVES. The u^e of Incentives as an Influencing factor on the efficiency 
of ul is relatgfkto the use of PCT In managing the progress of students In 
courses. The comments made In the section dealing with PCT are generally 
applicable here. 

There appeaNrs to be, 1 1 ttle consensus^among school personnel about what 
constitutes the ntost appropriate IncfientiVeK. for early course completion. 
Time off, extra leave/1 Iher^ty, and letters of commendation are in general use 
in NAVEDTRACOM but demonstrate varying levels of success. • It may be that the 
effectiveness of the incentives available to NAVEDTRACOM is directly related 
to the fact that such Incentives in othefsenvironments (nonmil 1 taryV are seen 
as rights and not privileges. JFurtKer, ci^umstances beyond the control of 
NAVEDTRACOM (geographic location, proximity to family, extracurricular activ- 
ities) may produce significant and confounding effects on incentive effec- 
tiveness. The kinds of incentives offered in the school house cannot cofnbat 
the disillusionment^ and low morale stemftiing from unrealistic expectations 
ab6Trt military life. Efforts to determine the relative effeotiveness of 
various types of incentives are just beginning, but the re^ulti of these ' 
efforts will still have to be applied in the atmosphere of reduced resource 
support for training. 

HOUSING AND MESSING. Like instruction, housing and messing availability are 
significantly affected by the peaks and valleys of course loading, resource 
allocation, and physical facilities limitations. 

Individualized instruction, because of its flexibil it>^ in start and 
finish times, may place added management duties on tj^^dministrative commarHl 
to ensure minimal disruption to the instructional p^cess. The following is* 
a list of potential problem areas identified: 

1. -Assignment of berthing spaces may not conform to shift assignments"* 
in multiple shift courses. 

l: Mess hall and other base facility hours are not always coordinated 
with shift assignment^. ^ 

3. Students may be required to change berthing even though their next 
school assignment is colocated. t 

4. Because of increase difficuUi.e3 in tracking students, additional 
regulations may be required to maintain good order and discipline. This 
problem is CQjnpounded by the loss of support services/instructors. , 

Major differences wepe noted ih the manner in which the authority/ 
responsibility for these functions was exercised. In some instances, the 
Base Command had overall responsibility for berthing and messing. In others, 
the School Command performed these functions. Local circumstances dictat«^ 
the relative efficiency of these- two approaches ^ 
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INSTRUCTOR ADMINISTRATION 

■» ' ^ ' 

This section Identifies two factors related to the role of Instructors 
In course administration whtch have not been /deal't with In previous sections. 

PLOWBA£K POLICY. Presently, graduates of "A" schools are being used to 
perform Instructional or instructional support roles. These personnel may be 
In a "hold" (medical, legal, etc.) status or reassigned directly to "A" school 
for-an 18 months Instructional, support role. There are a limited number of 
such billets. Individuals fill Ing these billets perform adhilnlstrati ve 
functions within schools and generally supplement reduced schoo/ staffs. 

This program Is scheduled for termination In FY 81 and the question of 
plowback replacements has not yet been resolved. If no replacements are 
assigned. Increased stress on assigned personnel and additional fractionation 
of instructor duties creating potentials for Inefficiencies Ifi the delivery 
of instruction can be expected. With the loss of this support to the Instruc- 
tional program, measures will have to be taken to replac.e this support If the. 
current quantity and qualify of instruction are to be maintained. 

COLLATERAL DUTIES. Discussions in several' previous sections have alluded to 
the increased requirements being plifced on instructors. These include: 

General Mil itary Training 

Administraj&iVe Support 

standing 

Ma^ter-ai-Arms (Barracks Watch) 

Course Dev^ppment 

Academic Review Board 

Disciplinary Boards. • 

The overall effect of these additional duties is to extend the normal 
workday, eroding what was perceived to be a benefit associated with instructor 
duty. This erosion makes instructor duty less desirable. Collateral duties 
are to be expected on a short-term basis; if they ocCur on a continuing basis 
they may have a deleterious effect ort instructors in an II environment. In 
this environoient, instructors are required to spend longer periods of time in 
the in5truptional setting with what are perceivedf to be less personally 
satisfying tasks than in CI. 

Several activities have suggested that, this additional load adds to the 
potential stress on* instructors as previously noted. In II this has led to 
greater "fatigue" and alienation effects. This is reflected irt decreased 
positive student-instructor relationships, stereotyped responses to questions, 
and, at some activities, promlon for rotation between instructional and . 
support duties on a regular basiS. The instructor training course under 
development ^ay provide at least a partial solution to these problems. 
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SECTION IV 



CONCLUSIONS AND RECOMMENDATIONS 



This section contains conclusions concerning general factors which may 
influence Navy technical training. It also provides recommendations relevant 
to the improvement of II in the Navy. ' ' 

CONCLUSIONS , ^ 



Ava^lable data strongly indicate the continued us.e of II as an instruc- 
tional strategy in Navy technical training. 



The remaining conclusions in this section are intended to convey a sense 
of the forces that are likely to be operating on all Navy technical training 
and, hence, shaping, its direction. They are organized into the general 
categories of trainijng effectiveness, instructional sti;ategies, instructional 
'management, manpower availability, and program administration. They are 
presented in no particular order of importance; however, the^r interactive 
characteristics should be carefully noted. 



Previous sections have identified a number of specific factors which may 
impact on the efficiency/effectiveness of II, Discussions of these factors 
have identified specific* problems and in many instances suggested solutions. 

TJIAINING EFFECTIVENESS. There will be a continuing emphasis on the need to 
achieve training effectiveness within limited resource availability. *This 
emphasis is currently reflected in 



requirements to develop management information systems for the 
evaluation of training effectiveness and to conduct effectiveness 
studies ./ 

requirements to develop and implement procedures and techniques for 
the implementation of recommendations stemming from internal and 
external evaluations 



INSTRUCTIONAL STRATEGIES. The use of II can be expected to continue and 
ejcpand in the Navy as increased emphasis is placedjon training efficiency. 
There is evidence that pipeline training times jire lengthening due to increasi 
complexity of technology. At the same time, resource support for training is 
not expected to keep pace with resource requirements. Thus, efficiency in 
training operations becomes a paramount concern. Since II reduces time in 
training with no apparent loss in training effectiveness, its continued use 
may be mandated by necessity. 

INSTRUCTIONAL MANAGEMENT. The management of *'inst ruction' via CMI^l play an 
increasingly important role within the NAVEDTRACOM. Computer raalfagement will 
be necessary to support the antici-pated growth in II programs. Increasing 
capabilities of "mini" and "micro" computers, significant advances In software 
and decreasing costs of hardware suggest that changes in' the configurations 




requirem(^nts to adapt and integrate new educational and hardware 
technology in current and future training systems. 
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of computers supporting Instruction can be expected. As* these new "systems 
are Introduced, policies and decision algorithms regarding the mix of cen- 
tralized and decentralized computer systems for training support will have to 
be' established. 

MAflPOWER AVAILABILITY. The size of the population from which Navy enlistees 
are drawn Is declining. At the same time, there appears to be a general 
Towering In the quality of these acce^ojg^ Unless acquisition policies 
change drastically, these trends will co|itiTme'*to have a significant Impact on 
training policy anW operations. That Is, Incoming personnel are likely to 
require more Instructor contact and/or remedial Instruction to prepw>e them 
for job-related training. This expansion of II may Include requTi>ements for 

provision for more flexibility In \ra1n1ng ^d testing 

an expansion of remediation- pro-ams to IncludeTjV-iielated academic 
material 



Increased emphasis on individualized student study programs, Includi 
study skills. - 



The requirements above have Imp'llcatlons for policies related to adminis- 
trative hold times, management of berthing and meSsIng, and most Importantly, 
the cost of training. 

PROGRAM ADMINISTRATION. Given the trends toward fiscal ausjterlty and Increasi 
competition for available resources, education and training requirements will 
have to be carefully documented and justified. Specifically, 

a closer coordination 6f the managemein>-dewlt>pment, delivery, and 
support aspects of the training pipeline will be necessary 

economic models appropriate to vai^Wus types of 'instructional 
decisions will have to be developed 

cost data bases and management informatfon systems appropriate to 
training efficiency analyses and the production of standard reports 
will have to be developed and/or refined. • 

RECOMMENDATIONS 




Recommendations for the improvement of II In the Navy and accompanying 
rationales are presented below. They are based on study findings and assume 
that II will continue to be used as an Instructional strategy In Navy techni- 
cal training. 

1. Establish a single office/activity with responsibility for all 
aspects of the integration and coordination of II Including Instructional 
development and Implementation, operational management of typertraining, 
management of centralized InstructlonaT development, and ADP support. Alter- 
natives to be considered In implementing this recommendatioji Include: 
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a. establish the responsibility with the Deputy Chief of Naval 
Education and Training or witn a staff code reporting directly to him ♦ 

b. appoint a "steering committee" composed of major participant 
activities, ch^1re<l by a nonparticipant as in la above 

c. assign the responsibility to an operating Assistant Chief of 
Staff (ACQS) within CNET. / 

It is further recommended that an interim office be established to- 
perform this function until such time as the .recommendation can be implemented 
on a permanent basis. 

2. Develop an information package to be presented to all' n)^EDTRACOM 
and. major fleet activities which would communicate the rationale, 'philosophy » 
and implementation procedures and policies associated with II. Examples of 
specific topics to be included in this package are 

command and organizational relationships 

definitions of terms 

feedback processes ^•"^^ 

external constraints. 

3. Initiate and support an effort to determine the relative effective- 
ness and efficiency of II for different kinds;' of training tasks and ability 
levels of trainees. 

4. Ensure the use of standard II terminology throughout the NAVEDTRACOM. 
These terms and definitions should make clear the distinction between instruc- 
tional strategies, instructional management systems, instructional delivery 
systems, and instructional media. UntlTsuch usage is generally prescribed 

. and accepted, confusion, complaints, a^lff^inappropriate criticisms may be 
anticipated. 

■ . ^ 

5. Develop and implement criteria for selecting among alternative 
instructional stra-tegies, instructional management systems , and/or instructional, 
media. 

6. Ensure that the training pipeline for II instructors includes 
materials appropriate to the role of the Learning Center Supervisor/Instructor. 

^ portion of this instruction may be devoted to material developed for 2 
above. In addition, this instruction should include topics such as 

computer operation in II • - 

testing limitations/alternatives in II 

student counseling 

course administration procedures ^ 
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progress monitoring 
coping with stress 
use of CM I reports. 



It Is further recommended that these materials be developed and Implemented 
on an Interim basis until such time as the Instructbr training cwrcicula being 
developed at Naval Education and Training Support Center (Atllantic) becomes 
available. ConslderatlonVhould also be given to greater standardization of 
current Instructor training offerings. 

7. Develop and Impleiment an II management course for all training 
administrator and school /course management personnel. A pdHlon of this 
course may be devoted to material as described In 6 above. Emphasis should 
be placed on topics of particular concern to management. 

8. Conduct a comprehensive survey to establish the types And extent of 
II In use throughout the Navy. - C*te<K>ries of Ins-truction In this analysis 
should.be based on the distinctions established In this report. 

9. Examine the desirability of providing preparatory materials in the 
use of computers in Instruction for students and/or Instructors. If determined 
to be appropriate, such programs might provide portions of the Interim training 
for instrufctors recommended in 6 above. Further, if such ttalninn is deemed 
desirable, available "off the shelf" packages should be examined for possible 
adoption. 

% ♦ 

TO. Establish a program to identify incentives and/or procedunis which 
act to improve student and Instructor performance in an II environment. Con- 
currently, conduct a cost/benefit, analysis of promising programs. 

11. Assess the relative cost benefits of alternative hardware systems 
for CMI. Considerations Of alternative student -input devices and centralized 
versus distributive processing should be Included in this assessment. 
Initial efforts in this area are underway. 

12. Develop procedures to locate, acquire, and/or develop cost data 
bases necessary for the conduct of the cost effectiveness analyses of alter- 
native training systems and apply the approach proposed in appendix B of this 
report. Apply thfs data as available to cost effectiveness comparisons of 
training approaches of Interest. 

13. Identify those data "elements found in the NITRAS. Navy CMI, training 
device, and, other management information systems which will support the 
monitoring and management of II in the Navy. Develop procedures to acquire . 
and maintain th1s.\^1nformat1on. 4 

14. Develop and Implement a management Information system fortthe 
management of Instructor personnel at individual training activities. Such 

a system should reside on currently available computer systems and^hould ^ 
include data elements such as ^ 
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Instructor qualifications 

past and present instructor assignments 

rotation assignments 

collateral assignments 

training assignments. 



B2 

57/58 



TAEG Report No. 78 



references" 



A Letter* Report ; 'A 'Comparative Evaluation of Computer-Managed and Instructor- 
ilanaged Instruction at the Propulsion Engineering School, Great Lakes, 
IL. 

Abramson, T. Evaluation of the New York City Computer Assisted Instruction 
Project In Elementary ArTthmetIc , Second Year , 1^69-70 - ITesearch Report 
70-16. October 1970. The City Unive/s1ty~arNew York, New York. N.Y. 

'0 

An Overview of the Ticcit Program . M74-1, January 1974. The Mitre Corporation, 
Washlng^n, DC. 

Ausburn, F. B. Impact of Learning Styles on Air Force TechnicaT^raining ; 
Multiple and Linear Imagery in tne PresentajTon of a Comparative Visual 
Location Task to Visual " and HaptTc Subjects . inFffRl^R-78-91 (11) , 
March 197^ tichnlcal TraTning Division, Lowry Air Force Base, CO. 80230. 

Ausburn , Lynna J . Impact of Learning Styles on Air Force Technical Training ; 
Relationships Among Cognitive Style" Factors and Perceptual Types . AFHRL- 
TR-78-91 (l). May 19797. Air Fqrce Human Resources Laboratory, Technical 
Train>Ti^Division, Lowry Air Force Base, CO. 80230. 

Back, Kathryn T., Stonsell, V., Ragan, L. J., Ausburn, F. B. , Ausburn, Lynna J., 
Huckaboy, K. Cognitiye Styles : A Bibl iography and Selected Annotations . 
AFHRL-TR-78-90 (ll). May 1979. ATr Force Human Sources Laboratory, 
Technical Training Division, Lowry Air Force Base, CO. 80230. 

Blaisdell, F. CAI in the Marine Corps , CAI Consultant, Navy Programs, Federal 
Systems DivTs"ion, San Diego, CA. Unpublished paper presented at the Sixth 
Naval Training Equipment Center/ Industry Conference, 15 November 1973. 

Braby, R., Henry, J. M., Parrish, W. Jr., and Swope^ W.M. A Technique for 
' Choosir\g Cost-Effecti ve Instructional Delivery Systems . TAE6 Report 

No. 16, April 1975, Revised October 1978. Training Analysis and Evaluation 
Group, Orlando, FL 32813. 

Brown, D. , Brandin, D. H., Cole, Phyllis, Marshall, T. H., Rubin, S. , Waksman, A 
Survey of the Use of Educational Technology i n the Armed Services . 
15 ApriT~l?7I. Stanford Research Institute,n^enTo Park, CA. (Prepared for 
Advan^d Research Projects Agency, 1400 Wilson Boulevard, Arlington, VA.) . 

Brudner, H. J. "Computer-Managed Instruction." Science , Vol, 162, 29 November 
1968, pp. 970-976. 

Cagle, M. W. "Dimensions." / TRA NAVY ^ September 1973. Chief pf Naval Education 
and Training, Pensacola, FL 32508, pp. 1-5. 

Carson, S. B., Graham, Linda L., Harding, L. G., Johnson, K. A., Mayo, G. D., 
Salop, Phyllis A. An Evaluation of Computer Manacjed Instruction in Navy 
Technical Training ,~Tr 75-38, May"T975. Navy Personnel Research and 
Development Center, San Diego, CA. 

59 

63 



TAEG Report No. 78 

REFERENCES (continued) \ 

Chambers, L. D. Managing a Transcontinental. CMI System . Point Paper, 4 
September 1979. NavaT Technical TralnlngTominand, Naval Air Station, 
Memphis. Mllllngton, TN. 38054 

i 

CNET Decision Memorandum No. 2, Subj: Establlshmeat of Instructional Program 
Development Centers. 27 April 1976. Chief of Naval Education arid 
Training (Code 00), Naval Air Station, Pensacola, FL 32508 



CNTT-A 10 Manual, Procedures for the Planning , Design , Development , and 

Management of Navy Technlcari"Tra1n1ng Courses . April 1^76. NavaT"Tech- 
nical Training Command, Memphis (75), Wllllngton, TN. 38054 (Superceded) 

• 

CNTT-A 67, Training Program Coordinator Handbook . July 1978. Chief of Naval 
Technical Training, Nav|.l Air Station, Memphis (75), MillingSon TN. 38054. 

CNTT-A 68, Catalogue of Naval Technical Training Publications . CNATT/CNTT 

Listing^ June 1577": CFief of Naval Technical Tra1n1ng\ Naval Air Station, 
Memphis (75), Mllllngton, TN 38054. 

Computer Managed Instruction iji the Navy ; A Comprehensive R&D Program . Navy 
Personnel Research and DeveTopment Center,\San Diego, W. 

ComputeVs and the Learning Society . Hearings Before the Sub-Committee on 

Domestic and International Scientific Plarfning, Analysis and Cooperation of 
the Committee on Science and Technology, U.S. House of Representatives, 
Ninety-Fifth Congress, First Session. October 1977, No. 47. 

Curry, T. F., Jr.^and Kincaid, J. P. Improving Your Navy Readinjj Skills. May 
197,9. Training Analysis and Evaluation Group, OrTand©, Fl. 32813. 



Dobrovolny, Jacqueline L., McCombs, Barbara L.,.and Judd, W. A. Study Skil ls 
Package : Development artti' Evaluation . AFHRL-TR-79-43, AugusFT579 
Technical Training DIvTsTon, A1r Force Humarv Resources Lab, LdWry Air 



Dobrovolny, Jacqueline L., McCombs, Barbara, L., and Judd, W. A. Time ^ 

Management Skill Training and Procedures for Increasing Student . * 
Efficiency in a Computer-M^maged Instructio nal •Environmenf . April 1979. 
McDonnell Douglas Astronautics Company, St. Lou-Is, MO. Paper Presented 
at the" Annual Meeting of the American Educational Research Association, 
San Francisco, CA. * 

Dobrovolny^ Jacqueline* L., McCombs, Barbara L., Judd, W. A. Orientation/ 
Time Management Skill Training Lesson : Development and Evaluation . 
AFHRL-TR-79-14. July 1979. Air Force Human ResourceiTaboratory, 
Technical Training Division, Lowry Air Force Base, CO. 80230. 

Doty, B. A. and Doty, L. A. "Programmed Instructional Effectiveness in Relation 
toXertdin St;ud6nt Characteristics." Journal of Educational Psychology/ 
1964, 55, 334-338. ' : — ~ — ^ 

60 ' ' 



TAEG Report No. 78 ^ 

REFERENCES (contthued) 

Economic Analysis Handbook . (2Dd Edition). Defense Economic Analysis Council. 

Farr, M. J. Generative Computer-Assisted Instruction ; Intelligent Instruction 
Intel! 1 gent Systems . Computers and CommunicatlorTS^ New York Academic 
lYess, Inc7 1977, PP. 407-469. 

Federico, P. and Landls.^D. B. Discriminating Between Failures and Graduates 

in a^ Computer-Managed Course Using Measu'reg of Cognitive Styles , Abilities , 
and Aptitudes . ■TR-79-21. June 19/9" Navy Personnel Research and 
Development Center, San Diego, CA. 

Federico, P. and Landis,. D. B. Predict ing Student Performance in a ComputeV- 
Manad ed Course Using Measures of Cqflnltiye Styley , AbilTtles , and 
Aptitudes , TR-79-30, August 1975^. fHiii^^ersonnel Research' and 
Development Center, San Diego, CA. ^ ' 

Fletchfifjp, J. D. Computer APPlicat;lon$ in Education and Training : Status and 
Trends . TR 75-32, Apri I- 1975^. i Navy Personnel Research ana Development 
Center, San Diego, CA. 

Frey, S. H., Shimabukuro, S. , and Woodruff, A. B. "Attitude Change in 
Programmed Instruction Related to Achievernent and Performance." 
AV Communication Review , 1967, 15^, 199-205 

Gagne, R. M. and Briggs^ L. Principles of Instructional Design . 
' " New York: iHolt, Rinehart and Winston, Inc. 1974, pp 22-25. 

Giunti, F. E. and Longo, A. A. An Interim Evaluation of the Tutorial Mode 
of CAI in Army Basic Electronics Training . Computer Assisted 
TnstructTon Division, U. S Army Signal Center and School, Fort Monmouth, 
NJ. Paper presented at the Annual Convention of the Northeastern 
Educational Research Association, Grossinger's, Grossinger, NY, ' 
November 9-12, 1971 , V, 

Goldman, H. D. Group-Paced Presentation of Programmed Instruction . ^ Evaluation 
Project Report NR. 65-2, 1 November T5^64. Amarillo Technical Training 
Center, Air Training Command, Amarillo Air Force Base, TX. 

Goldman, H. Air Training Coninand Headquarters, Randolph Air Force.Ba.se, TX, , 
Personal Communications, 1979 

Hall, K. A. Inservice Mathematics Education Via Computer-Assisted Ins-truction 
for Elementary School Teachers in AppaTachia . Rfeport No. R-^7, January 
1970. The Pennsylvania State University, University Park, PA. 

Hansen, D. N., Ross, S. M., Bowman, H. L., Thurmond j. P. Navy Computer Managed 



Instruction : Past , Present , and Future . July 1975. Bure.ali of Educational 
Research and 
Memphis, TN. 



Research and Services, Foundations of. Education, Memphis State^ni versity , 



!' ■ 65 



TAEdf Report No, 78^. ' • 

REFERENCES (dntlnued) 

Harding, L. 6. and Flelschtnan, H. L. Aptitude Level and Consumer Acceptjncg ; 
of Pro grammed Instruction . Research Memorandum 68- 9 . Oc tober 1 §67 . ^ 
/ D7 S. Naval Personnel Research Activity, San Diego, CA. 

Hodak, G'. W. , Middl'eton, M. G., Rankin, W. C. Institutionalization of 

Instructional System Developmerit (ISD) in the Naval Educa t ionn aHg' Training 
Command ; An Organ^2atiQna1 Ahalysjs . TiHTG^l^port No. 70, May T5T9. 
Training Analysis and Evaluation Group, Orlando, FL. 

dak, G. W. , Middleton, M. G., Rankin, W. C. Papetti, C. J, Personnel . 
Attrition from Navy Enlisted Initial Technical Training . TAEG Report 
No. 69, March 1 979. Training Analysis and Ev^l uation Group , Orl ando , Ft. * 



Individual izlng learning . Bibliographies . Research for Better Schools^ Inc. 
Suite 1706/1700 Market Street, Philadelphia, PA.. 

Integrated DoD Plan for R&D on Computers in Education and Training, September 
1975. • ■ . ' 

Kearsley, G.' P., Romaniuk, E. W. , Rowand, J. Computer Based . Guj^gance : The 

Shape of Things To Come, Research and Information Report .RIR-77-3, January 
1977. Division oT Educational Research Services, University of Alberta, 
Edmonton, Cafiada. 

Kearsley, G. P. Some' 'Facts' About CAI :-' Trends 1970-1^76 : 

King, Anne Tru$cott.*^ Impact of Computer-Based Instruction 6n Attitudes of 
Students and Instructors : A Review . AfHRL-TR->57.4 , May 1975. Air 
Force Human Resources Laboratory, Technical Training Division,: Lowry Air 
Force Base, CO. 80230 • 

Knerr, B. W. Relationships Between Student At titudes Toward Computer-Assisted 

Instruction and Training Pe rTormance . Technical Paper 135, August 1978. 

U. S. Army Research Institute for th^ Behavioral and Social Sciences, 
Alexandria, VA. 



Kopstein, F. F. and Seidel, R. J. v Compttter-A'dministered Instruction versus 
Traditionally Administered Instruction : fefconomics " ('rofessionaT*^ 

imj^n 



Paper 31-67, June 1967. Humin Resources Research Office, Alexandria, VA. 



Kottenst^tte, J. P. Microfiche Applications in an Individualized , Self -Paced 
. Learning System . AFHRL-TR^79-6, May 197^7 "TTir Force Human Resources 
Laboratory; Technils^al Training Division, Lowry Air Force Base, CO. 80230. 

Lintz, L. M., Pennell, R., and "Yasutake, J. Y. Integrated System Test of the 
Advanced Instructional System (AIS). AFHRL-TR-79-40, August 1979. 
Air Force Human Resources Laboratory, Technical Training Oivision, Lowry 
Air Force Base, CO.- 80230 



TAEG Report No. 78 



REFERENCES (continued) 



Lockhart. Kathl^emA. Component Research in Personalized Systems of 

Instruction ImplicaTTonsn Por Tiojnp^ Military Technical Training , 

Technical Note, May T979.*nTavy P«rsbnn«l Research and DeveloNent fienter, 
San Diego, CA. ' , 

Lumsdalne, A. A. and Glaser R. (.Ed) Teaching Machines and Pr^ogrammed Learning : 
A Source Book . Washington^ DCt Nqtlopfl EducatlorTTCssociatldn, 1960. 

Mathis, A., S^tiith, T., and Hansen, D.^ '•Colleige Student's Attitudes Toward 

Computer- Ass is ted Instruction." Jourrral of Educational Psychol ogyt 1970, 
61, 46-51. z: . ' 



McCombs, Barbara 
Instruction 



McCombs, Barbara L. I den t i fy i ng I ndi v i dual ixa ti on Paf'ame ters , Strategies and 
Models That Are User Accep tab] e_ and ^easi b1 e ; Methodological Considera- 
tions . ~RpriTT97^ McDonnell Douglas ^itronau tics Company, St. Louis, 
MO. Paper presented at the Annual M««t-1hfl of the American Educational 
Research Asso., San Francisco, CA.*" . 

L., Dobrovolny, Jacqueline,, Judd. W. A. Computer-Managed 

. Develbpment and Evafua^tiQJj of Student SkU\ Modules to: 

Reduce Trafning Time .~~RrHRraRV9«2b;-^3;fltjgMit 1979. Air Force Human 
Resources Laboratory, Technical Training, Division, Lowry Air FOrce Base, 
. CO. 80230. ■ • . ' . , , • ^ 

Middleton, M. G., Papetti , C. J., Mifcheli, Gi. S. Computer Managed Instruction 
in Navy Training . TAEG Report No. 14, March T^TJT Training Analysis * 
and Evaluation Sroup, Orlando, FL 32813 . ^ 

Mitzel, H. E. A Commonwealth Consortium to Develop , Implement, -and Eyalujite 
3 Pilot Pro gram of Computer-Assisted"Tnstructjon ^ for» Urban HTg)i Schools . 
Ifeport No. R-47,'~July 1971, The Pennsylvania State University, CAI 7^ • 
LaboiratoK^, University Park, PA. H\ 

Montemerlo, M. D. Instructional Systems Development , State of the Art and 

Directions for the Future , .Human Factors Laboratory, NavalTraTnTng^rquip- 
ment Center, Orlando, FL. ' ' 

Moore, M. V. and Nawrocki, L. H^ The Educational Effedtiveri egs of Graphic 

Displays for Computer A%s is teHnTns true t ion . Technical {B|er 332, September 
1978. Unit Training and Evaluation 'Systems -Technical Ar^, u. 5. Army 
Research Institute for the Behavioral and Social Sciences, Alexandria, VA, 

^ ■ ... 1b 

NAVEDTRA 106A, Interservlce Procedures for Instructional Systems Development 
( 5 Vol umesTT August 1975r; Chief o7~Naval EducarHQn and Training, . 
Naval Air station, Pensacola, FL 32508 

- ■• . . . ' 

NAVEDTRA 108, A Technique for Choosing Cost- Effective InstlructioTlal Del 1 very 
Systems , April 1976. Chief of Naval Education and Training, Naval Air 
• Station, Pensacola, FL 32508 \ ' ' ' 



»j TAE6 Repqrt No. 78 

REFERENCES (contli^ued) . . ' . 

t 

♦ 

NAVEDTRA HQ, Procedures for Instructional Systems Development . July 1978. . . 
Chief of Naval tducatloh and Training. Naval Air Station. Pensacola. FL 
-32508. . 

NAVPERS 93510-1. Fundamentals of Nav^y Curriculum' Planning . October 1968. 
Bureau of Naval Personnel ."Washington. DC. 

Nelson. C. B. The Effectiveness of the Use of Progr^hmed Analyses of Musical 
Works on Student's Perception oT Torm . Tinal Report. 196/. Estate 
UniveriTty of New York at CortTand. NY. 

Northrop Corp. Future Undergraduate Pilot Training System Study . Final Report 
Appendix XI : A Review and Application of Instructional Concepts and 

" jL Techniques . N^fR 70-149. 1971. Aeronautical Systems Division. WrTght- 
Patterson AFB. OH. (AD 881860). 

O'Neal, F.. "The Role of Public Schools in Computer Assisted Instruction." 
Educational Technology . March 1970. 5-10. ^ ' 

Orlansky. J. and String. J. CoH- Effectiveness of CompKjber- Based Instruction 
ijl Military Training . IDA Paper P-1375'> AprTT 1979. Institutfi^r 
Defense Analyses. Science and Technology Division. Arlington. VA,' 

Ragan, T. J.. Back. Kathryn T.. Stansel 1 if V. . Ausburn. Lynna J.. Ausburp. F. B.. 
Butler. Patricia A.. Huckaboy. K. Co gnitive Styles : A Review OT the 
Literature . AFHRL-TR-78-90 (1). May 1979. Air. Force Human Resources ' 

' . Laboratory. Technical Training Division. Lowry Air Force Base. CO 80230. , 

"* * < 

Report of the Joint Technical Coordinating Group Members of -Subpanel on Computer 
, Assisted Instruction (CAI) and Computer Managed Instruction (CMI). DRCPM- 
' '-TND-SE. 12 February 1979. Headquarters U. S. Army Materiel' Development 
..arid Readiness Command. Office of the Project Manager for trai.ning Devices. 
Naval Training Equipment Center. Orlando. FL 32813. ^ 

Response of the Navy. Response to the Summary Report of the Task Force on 
Training Technology. Defense Science BoaI'd. Department of the Navy. 
30 August 1976. . . , • • 

Robi/»sqn. T.\ E. and Lautenschlager. Eleanor. Preliminary Evaluation Study : 
Measuring Computer-Assisted Instruction Effectiveness in Algebra- U. 
July 1971. Computer-Assisted Instruction Project. Montgomery County 
Schools. • ,* , ' 

■ '* - ' ' ' . ■ " . .' 

Rgth. R. H.* "Student Reactions to Programmed Learning." Phi Delta Kappan . 

- 1963..44. 278r281. v . ' . • 



ftundquist. E. A. Cour?se Design andfRedesigri" . Manual for' Job Training Courses , 
(First Edition).* S.RR 66-17"TRevtsed).. January 1967. U. S. Naval 
PersonK^lvSesearch Activity- San Diego., CA 92152 




TAEG Report No. 78 



REFERENCES (continued) 



Sachar. Jane. Abrams. Macy and Buckley. Christie. ReTatina Pecformance in 

Basic Electricity and Electronics and ''A'SSchools\ TR 79-2. October 1978. 
Navy Personnel Research Ajnd DevelopmenlTenter. San Diego. CA. 

Seldel. R. J.. Compton. Judy G.. Kopstein. F. F... Rosenblatt. R. D.. See. Sally. 
P roject IMPACT : Description of Learning and - Prescription ^oy'^^"Str"<^t1on,' 
TWeH'lonaTTaper 2Z-69. June~l9B^ RumarTftesources Research Office . 
Alexandria. VA. 

Seidel. R. J. and the IMPACf Staff. Project IMPACT . Computer-Administered 
I nstruction : Preparing and Managing the Content of Instfy-uctlorT , IMPACT 
Text'-Handl fng Subsystem .~Tepteniber 1?7T. Human l^esources Research 
Organization. 300 North Washington Street. Alexandria. yA. • 

Seidel. R. J. Project IMPACT : Computer-Administered instruction Concepts and 
Initial DevelopmenT : Technical Report 69-3. 'March 1969. Human Resources 
Research Office. Alexandria. VA. 

Seidel. R. J. atid Wagner. H. Computer Based Training^ and Educational Man ageirient 
Aspects . Unpublished document to ap{i^ In HaroldF. O'Nell. Jr.. (Ed.). 
State-of-the-Art Assessment in Computer^ased Instruction, Academic Press. 

Self- Paced Instruction ; An Evaluation of. Memorandum from Commanding Offfcec. 
RavaT Technical Training Center. Corry Station. Pensacola. FL. to Chief . 
of Naval Technical Training. 17 February 1977. , 

Skinner. B. F. The Technology of Teaching . New York': Appleton-Century-Crofts. 
1958. pp 30-32. , 

Splittgerber. F. L. J'Computer-Based Instruqtion; A Revolution in the Making?" 
•Educational Tech'^nology . Jan.uar^ 1979-. pp 20-35. 

Summary Report of the Task Force oa Training Technology. 27 February 1976. 
Defense Science Board. Of free 'of t|ie Director of Defense Research and 
. Engineering. Washifigton. DC. • 

Swope. W. M and Green. E. K. A Guidebook . for Econoinic Analysis in the Naval 

Education and Training Command . , TAEG Report No. 55. April T^lWT Trarining 
Analysis and Evaluation Group. Orlando.. FL. 32813 

Van Matre. N. Navy Computer Managed Instruction Res(|arch : Background . Problems 
and Res earcTTTs sues .' Navy Personnel Research and Development Center . 
ego. CA. (Briefing given at CNET Headquarters on 12 June 1979.) 

Wolfe. J. H. aiid Wil 1 jiams . ,M. D^ Prospects fpr Low-Cost Advanced Computer^B^sed 
. Training : " A Forecast . Special' Report 79-16. Apri 1 1979, San Diego, CA. 

2ech, L. W.. Jr. Se}f-Paced> Inltruttion ; An Evaluatiwi of! "Memorandum from 
Commanding Officer. Naval Technical Training Center.- Cpr.Ky Station. ' 
Pensacola^ FL to dhief t)f Naval'Technical T^-aining; 17 February 1977. 



•TAE6 Report No. 78 • 

1 ' • - 

ADDITIONAL REFERENCES . . 

• " ' ' ' , , i * 

CNET INSTRUCTIONS / , ' ^ / * ' r 

1500.12, *with Change 1, Glossary of Navy Education and Training Terminology. 
14 September 1976. 

1550.1 A, Instructional Systems Development. 6 January 1976. * 

1550.5. Instructional Systems Developm6nt;»Pol icy and Doctrine for. 11 March 1974. 

1550. 13A. Instructional Program Development; Responsibilities and Approval, 
Procedures. 25 July 1978. *• 

5231 1. with Change 1. Naval Training Information System (NAVTIS). Automated 
*Oata System (ADS). Development and Operation; Procedures for the Computer 
Managed Instruction (CMI) System. 17 February 1977. 

5260.1, Computer Managed Instruction (CMI) System. 17 February 1977. 
5311 .1 . Specialized Training Staffing Requirements; Computations of. 18 July 19 
STAFFINST 5400. IC. Staff Organization Manual. 30 December 1977. 
STAFFNOJE 5400. Staff Organization Realignment. 5 September 1979. 
CNTECHTRA INSTRUCTIONS . ^ 

1540.40. Curriculum and Instructional Standards Office (CISO). 3 January 1979. 
1 540. 44. Student Reporting and Record Keeping. 5 July 1979. . 
5400. 7A. Computer Managed Instruction. 22 May 1974. " 
NOTE 1500. Guidelines for the Use of thfe CMI System; Promulgation of . .23 June 1976 

OTHER / • 

DOdInS? 7041.3. Economic Analysis and Program Evarlua t ion for Resources . 
Management, 18 October 1972. ^ 

NAtTCMF^IN^^ 1540.19, NATTC Memphis Student Advisor Program, 21 June 1979.. 



/ 



^ 



TAEG Report No. 78 

^ APPENdlX A 

LtST OF COMMANDS AND ACTIVITIES 
VISITED/CONTACTED 



U.S. Army 

Army Research Institute, Alexandria, VA ^ 

Signal School, Ft. Gordon, GA ^ , ^ r. * r«««,«n^ 

Training Development Institute, U.S. Army TrainiT^g and Doctrine Command. 

Ft. Monroe and Ft. Eustis, VA 



/ 



U.S. Air Force 

Mf- Training Command Headquarters, JWfolph AFB, fx 
*Human Resources Laboratory, Lowry AFB, »^ CO 
Office of ^cierltific Research, Boiling AFB, Washington, DC 



Pilot Instructor Training Schdol , Raodolph AFB, TX 
3270th technical Trailing Group, Lackland AFB, TX 

y 



\ 



U.^. Navy 

Aviation Mechanic "A" Schbol , NAS Memphis, Millington, TN 
Aviation F'undamentals- (P) Course, NAS Memphis, Mil 1 ington. TN - - 
Avionics Technician "A" School. NAS Memphis. Millington. TN 

BE&E School, NTC Great Lakes, IL , 
BE&E' School. NAS Memphis, Millington, TN 

BE&E School. NTC Orlando, FL 

BE&E School, NTC San Diego, CA r. ^ 

Giiief of Naval Education and Training, NA5* Pensacola, FL 
Chief of Naval Technical Training, NAS Memphis, Millington,, TN- 
Instructional Program /Development Center, NTC Great Lal:es, I L 
Instructional Progranjf DevelopmWit Center, NTC San Diego, CA 
Instruc/or Training School, NAS Memphis\Mill ington, TN 
• Mahageiient Information and Instructional ^sterns Activ^ity, Pensacola, FL 
h\r Technical Training Center, NAS Memphi/s. Millington," TN 
^NavaT Educatiqn and Traininfl Support Center, Atlantic, Norfolk, VA 
Navy Personhel Research and Development Center, San Di^go, CA 
Office of Naval* Research, Arlington^ VA 

Propulsion Engineering (Bas'ic) School, WTC Qreat Lakes, IL* . 
Radioman "A"' School, NTC San'Diego^CA 

Service School Command, NT-C^OrlandO, FL , X <^ • 

Service School Cdwnand, NTC Great Lakes, IL 
. Service Scheol Command, NTC San Diego, CA ; 



ERIC 



♦Contacted 



67/68 



7i' 



TAEG Report No. 78 
APPENDIX B 



0 

A MODEL FOR ECONOMIC ANALYSIS OF 
\ INSTRUCTIONAL STRATEGIES 

At -some point during the course development process, the educator just X 
choose between Individual and group strategies, between the various media* 
and between Instructor and computer management. It Is Hkely that seyeca4^ 
different kinds of courses could traifi a given set of tasks with equaiveffec- 
tlveness; i.e.. the degree of training, as exhibited tn end of course tdsts^ 
would be equal for different kinds of instruction. Consequently*, the decision 
on the kind of course to be /level oped must depend on some criteria other than 
effectiveness. Given such Choices. DOD policy dictates that resource costs 
will ^ the criteria; the alternative w1thf the lowest life-cycle cost will be 
the one selected for implementation. / 

OVERVIEW OF "COST-EFFECTIVENESS" ANALYSIS ^ ^ 

The appropriate' approach to use tn comparing alternative training (ystems. 
In order to determine which would be least costly.' 1s "cost-effectlvenesS - 
analysis. Cost-effect1vene^s analysis 1s the most widely used term but- It Is 
synonymous with "economic analysis." the Office of Manpower and Budget s 
"cost comparison analysis." the corporate financier's "c^|)1tal budgeting 
analysis." and the defense analyst's "life-cycle costing." Regardless of 
name, the methodology remains essentially the same and the dec1s1on-mak1ng 
solutions are Identical. "How-to" Instructions abound— DOD Instruction 
7041 3 "Economic Analysis and Program Evaluation for Resource Management and 
the Defense Economic Analysis Council's "Economic Apalysis Handbook' provide • 
general Instructions for cost-effectiveness analyses. TAEG Report No. 55. A 
-£u1debook for Economic Analysis In the Naval Education and Training Command, 
provides more detailed Instructions, while appendix B toTAEG Report No. 16, 
A Technique for thoosing Co^t-Effective Instructional Delivery Systems, 
contains an WcosTmodel which would be most helpful when conducting training 
cost-effectiveriess analysis. In the finaPanalysis. these approaches are 
.nothing, more than finding the cheapest waV to do something. 

In general, the co^t-effectivepess analysis involves summing the relevant 
costs for each Jilternattve and awarding ihe decision to the least costly 
option. The issue of relevancy is paranlount. Costs are relevant only if 
•they will occur jn the future (Ue.. arfe not yet "sunk" and if they are 
variable) (i.e., vary among the alternatives being considered)). 

I ' " * 

For example, assume that a new course is being developed •to train electronic 
switchboard operators. The educatgr has determined that three training alterna- 
■tlves will do the task equally well. They are a computer managed self-paced 
course, an instructor managed self-pct«^ed cpurse, and a course using conventional 
' instruction. ' The conventional ihstruction would' use lectures and texts, while 
'the self-paced iiistruction would use programmed texts. The cost analyst's task 
'would be to look at each individual productive resoprce and to estimate the 
amount that would have to be purchased or diverted from oth6r organizations 
in onder to accomplish the training mi^sjon. A hypothetical summary cdst - ( 
sheet for the above three alternatives appears in table B-1. « 
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TABLE B-l HYPOTHETICAL COST-EFFECTIVENESS STUDY THREE^LTERNATIVt 
SWITCHBOARD OPERATOR COURSES 



l^esource Category^ 
Course Development 
J Course Operation 

A. Student Compensation 

"ctructor CompensatitfrT^ 
Classroom Modifications 



Life-Cytle Relevant GoSts for Three 
Training Systems (millions $) 

IMI . Conventional 
Self-paced Instruct1o.p 



.CMI 
Sen-paced 



$ 4.0 



$ 3.0 



B. 
C. 
D. 



Supplies Equipment^ 
/ Conventional Texts 



«P/I Texts 
Computer 



/ 



Total 



?1.0 


24.0 


30.0 


.7 V 


.8 ' 


1.0 


.3 


.1 


0 


* 

0 


0 


:i 


.1 


.1 


0 


'.2 


0 


D 


$26.6 


$29.' 


$34.1 



lrAto«<riP^ are adaStatlons of categories found In the TECEP cost model, 
apSii B to TAEg'S>No. 16. A Technique for Choosina Cost-Effect jve 
Instructional Delivery S ystems . ^ 
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Included in the hypothetical analysis are only those cost^ which vary 
between alternatives. For example, since health care, base support functions, 
personnel support activ,i ties , student travel, and other similar costs are not 
included, it must be assumed that they apply equally to the three alternatives 
being evaluated. If in reality such expenditures did vary across the alterna- 
tives, they would then have to be included in the study. In this example, 
the CMI system has the lowest cost and should therefore be chosen for 
implementation. 

PROBLEMS AND RECOMMENDATIONS 

RAW COST DATA. Raw cost data is generally available for all categories 
except course development costs. Until the recent advent of the IPDCs, no 
data were retained which could inform the analyst how much labor, equipment, 
or supplies were expended in the development of specific courses. However, 
the existence of an accounting system for IPDCs may remedy this difficulty. 
As the centers come down their learning curves and their costs moderate, 
analysts will have available an excellent source for development cost estimates 
To insure complete developmental cost data which can be tracked to the course 
being developed, it is recommended that IPDC managers account for the hours 
of effort expended by their employees on courses being developed. 

PROJECTING TEACHING/STAFF RATIOS AND STUDENT COURSE TIME. Note in the hypo- 
thetical example that the "student- compensation" was greatest for conventional 
instruction, 20'perc€nt less for instructor managed individualized instruction, 
and 30 percent less for computer managed individualized instruction. This is 
based on the fact that most research shows considerable savings in students' 
learning time when individual ize(kinstruction replaces conventional iTistruction 
some research has indicated a further savings when a self-paced course is 
converted from instructor managed to computer managed. 

The problem is 'that some of the research is sketchy and controversial. 
Therefore, the cost analyst should (1) keep abreast of new research as it 
evolves and (2) seek and carefully consider professional educators' opinions 
on what they believe will be the actual time^savings for the course being 
analyzed. ^ ^ 

Note also that in ,the example the staff costs decreased proportionally 
with student costs; i.e". , the student/staff ratios were constant for all 
thr^ alternative training systems. Again, these ratios are a subject of 
some controversy in )»esearch and in managerial guidance. Therefore, t>he 
analyst must again consider current research and current managerial policies 
when evaluating staff requirements. 

COST ANALYST EXPERTISE. All the "how-to" instructions for cost analysis 
contain one common caveat--the analyst must account for all the relevant 
economic costs of the resources. In many' instances , the economic costs are 
equivalent to the purchase price of the item. However, in other instances 
the economic costs have no relationship to the purchase price, and there- 
fore must be valuated by the analyst. 
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For example, assume that a tralStng center owns a large central computer 
with much unused capacity and that ithes been determined that this capacity 
»w111 be reserved solely for future CMI use. If one were to perform a cost- 
effective analysis today in which CMI was being compared with IMI. the computer 
costs for IMI would be zero (since no computer 1$ use^ and would also be 
zero for CMI because idle Computer spacers being reserved for its use. The 
zero costing for the computer in this situation is totally compatible with 
sound, accepted managerial practice. Recall that one only counts relevant , 
costs and that relevant costs are future costs; i.e.. not yet sunk. Jhe 
co^nputer in this example. was purchased 1n the past and Is being, reserved for 
CMI use; therefore, no future costs will result fromrUs use tomorrow. 
.However, given the same situation except for the fact that the excess computer 
capacity can now be used by other activlities and that there are other activi- 
ties waiting to use It. the computer woilld have to be yaluated in the CMI 
alternative at approximately today's replacement value (no% at the past 
purchase price). 

» ' 

Economic analyses are situational. expeciaTTy when one is dealing with . 
long-term capital expenditures such as computers and facilities. Such pro- 
posed purchases should be evaluated by people with sufficient expertise to 
determine their true economic fcosts if meaningful and correct cost analyses 
are desired. 

♦ * 

MAJOR ECONOMIC TRADE-OFFS j 

Unfortunately, as evidenced by the preceding discussion, very few con- 
crete, irrefutable rules can be \mafle about efficiency and teaching strategies. 
One cannot say that "all courses with AOBs greater than X should be individ- 
. ualized and those with less should be taught conventionally." One can only 
say "the ultimate c'ourse strategy should be determined by relevant costs, and 
relevant Costs depend &n the relativ'e costs of the productive resources used 
in the course." 

However, major trade-offs can be identified which might help in strategic 
managerial planning: 

CMI VS. IMI., An indiv'idualized course can be either computer managed or 
instructor managed. This fs the old issue of whether to' automate or do some- 
thing manually. As ADP costs become lower, vis-a-vis personnel costs, Qne 
might expect the use of computer management to become increasingly attractive. 

INDIVIDUALIZED VS. CONVENTIONAL STRATEGIES. Research indicates that individ- 
ualized Instruction saves at least 20 percent in student time or salaries / 
.paid, when compared to conventional instruction. ' Since student salaries 
are the largest costs injthe total training budget, even small percentage 
savings in student time can lead to appreciable dollar savini^s. i 

However, individualization is not a free good. It is more expensive to 
develop the course and to manage the students in individualized instruction 
than in conventional instruction.* Therefore, the savings in student tinle • 
must be carefully weighed against increased developiftent and student manage- 
ment costs which evolve from the individualized strategy.' The equilibrium 
point of the trade-off must be thiat point where the relevant costs are 
minimized. ' ■ ' . . 

ERIC , • / a 
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CONCLUSION 

Given the real world situation of ever tightening availability of 
resources, it becomes increasingly necessary for the Navy to get "more" for 
"less." Educators must continually search for methods which will effectively 
train the service's people. Once these methods are established, the decision 
on which to implement must be made on the basis of costs, since costs are the 
only measurement one has of the relative scarcity of ptoActive resources. 
Only by insuring the most cost effective means of operation can the Navy get 
the most training from the resources it is given. 
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APPENDIX C 

r 

A LISTING OF INDIVIDUALIZED COURSES 
CURRENTLY SHOWN ON NITRAS 



Method of Instruction (MI) code: 

'I 

B - Both self-paced and computer managed 
C ■ Computer managed only 
P ■ Self-paced only 
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